CSV Filter - Backslash double quote parse failure

Elastic Stack Logstash
1

Hi,

I am using version 7.12.0
I'm parsing a csv which looks like this:

colA;colB;colC
x;"hello \"world\" ";0
y;"hello world";1

I cannot parse the x row because I get the following:

Error parsing csv {:field=>"message", :source=>"x;\"hello \\\"world\\\" \";0", :exception=>#<CSV::MalformedCSVError: Missing or stray quote in line 1>}

So I was thinking about removing the \" altogether, thus my filter looks like this:

filter {
  csv {
    autodetect_column_names => "true"
    separator => ";"
    skip_header => "true"
  }
 mutate {
    gsub => [ "colB", '\\"', '' ]
 }
}

I have tried many other variants (", \", \\", ...) but I am still getting the same error.

Can someone support?
Changing the source file/format is not a viable solution.

Thank you

2

Ok, I figure it out.
The issue happens BEFORE parsing, so in my case applying a substitution to the whole string works:

filter {
  mutate {
    gsub => [ "message", '\\\"', '' ]
  }
  csv {
    autodetect_column_names => "true"
    separator => ";"
    skip_header => "true"
  }
}
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.