Removing text qualifier double quotes from Logstash CSV output

Elastic Stack Logstash
1

Not sure if possible but I'm creating a CSV using Logstash. When I open the CSV in Notepad++ it adds double quotations around one specific field. The field itself is a city state zip code field that is created using a mutate filter.

So for example, in the CSV, the field appears as:

New York City NY 10009

There are no quotation marks in the text itself. But opening the CSV in Notepad++ it looks like:

"New York City NY 10009"

Is there anything in my Logstash pipeline I can do to keep those quotation marks from being added or is it just a limit of CSV text qualifiers? The quotation marks are also not present when ingested into Elastic.

2

The only time I would expect a csv output to add double quotes around a field is if that field contains a comma.

1 Like
3

Thank you for your reply. Got it figured out and you were close. So the mutate was using a GREEDYDATA pattern to create the Zip Code field that was at the end of each original line of data. What was happening was it was including the line break in the field. I switched it from GREEDYDATA to USERNAME and now it's not including the line break and thus the CSV isn't adding the quotation marks. Weird, but I guess it worked!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.