Logstash comma delimited csv file with quoted strings

Elastic Stack Logstash
1

Hi,

I have following comma delimited csv file, which content has a comma in some of the columns.

LogTime,Level,ApplicationId,UserId,ProcessId,FormId,Action,Status,Reference,TranRef,IP,SessionId,CIF_REF_NO,UserAgent
11-12-2020 00:00:00,INF,Investor,"2D5D066C-DE96-41D9-8F49-B0F2211D46FF","FIRSTTIME_LOGIN","FIRSTTIME_LOGIN",RequestPAC,S,"Uid:CII0700000242495,",,118.101.247.211,sf3wkj25s1udkn5fynn5ikj0,CII0700000242495,"Mozilla/5.0 (Linux; Android 10; LYA-L29 Build/HUAWEILYA-L29; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/87.0.4280.101 Mobile Safari/537.36",""

Error message when i run it in logstash:

Error parsing csv.... :exception=>#<CSV::MalformedCSVError: Missing or stray quote in line 1>}

I am reading the file with following:

filter {
  csv {
    separator => ","
    columns => ["LogTime","Level","ApplicationId","UserId","ProcessId","FormId","Action","Status","Reference","TranRef","IP","SessionId","CIF_REF_NO","UserAgent"]    
    skip_empty_columns => "true"
  }

May i know how should i deal with these type of csv files ?
*note: i can't change the source to use different delimiter.

Regards.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.