exception=>#<CSV::MalformedCSVError: Unclosed quoted field on line 1

[vikas_gopal]

Hi Experts,

I am getting exception=>#<CSV::MalformedCSVError: Unclosed quoted field on line 1 error while parsing a CSV file. Here is my data looks like . I am getting comma separated values surrounded by double quotes

May 30 22:59:55 10.0.0.0 "4","0","","","Authentication/Verify","Operating System"

in logstash5.4 I am defining column names and replacing double quotes with single quote because values were getting ignored if they are blank . I thought I parsed the whole csv like 80% but now i checked the logs and I am getting above error , not sure where it is failing .

my LS configuration

filter {
grok { match=>["message",'%{CEFDate:sysdate}\s%{cefip:sysip}\s%{CEFMSG:endmessage}'] }
mutate {
gsub => ["message","\"","'"]
}
csv {
        source => "endmessage"
autogenerate_column_names => false
 columns =>["baseCount","AC","bytesIn","bytesOut","behavior","dvcgroup"]
}}

Not sure if this problem is because of empty value (byteIn, byteOut) or a white space in dvcgroup (Operating System). I do not found any solution , can someone help me to understand what could be the problem .

[vikas_gopal]

Never Mind ,

So I got the problem it was with my data . So I had mentioned that I was able to parse more than 80% of data with above config . Rest 20 % was not in proper csv format which causes the problem . I exclude those and things works . So no more errors.