# Problem with quotes in csv filter
**URL:** https://discuss.elastic.co/t/problem-with-quotes-in-csv-filter/382802
**Category:** Logstash
**Created:** [October 17, 2025, 10:40am UTC](https://discuss.elastic.co/t/problem-with-quotes-in-csv-filter/382802 "2025-10-17T10:40:17Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@parosio](https://discuss.elastic.co/u/parosio)
#### Post date: [October 17, 2025, 10:40am UTC](https://discuss.elastic.co/t/problem-with-quotes-in-csv-filter/382802/1 "2025-10-17T10:40:17Z")
Hello,
I'm having a problem with the csv plugin (seems the same problem as in [this old (2018) article](https://discuss.elastic.co/t/csv-filter-quote-character-parse-failure/116611))
Quoted strings within fields that do not contain the field separator are interpreted as fields, and this provokes a parsing error.
For example, this input line:
```auto
2025-10-16 01:44:55,"style :expression (" (Parameter),5,64.39.106.92,10.97.203.236,/Common/dem-www-pr_443,Cross Site Scripting (XSS),1.0
```
raises the following error:
```auto
Error parsing csv {:field=>"message", :source=>"2025-10-16 01:44:55,\"style :expression (\" (Parameter),5,64.39.106.92,10.97.203.236,/Common/dem-www-pr_443,Cross Site Scripting (XSS),1.0", :exception=>#
```
If I set a different char as quote\_char:
```auto
csv {
columns => ["Date_Time", "Signature_Name", "Severity_Id", "Source_Ip", "Destination_Ip", "Policy", "Attack_Type", "Hit_Counter"]
separator => ","
quote_char => '§'
}
```
records with fields that contain the separator character, and are therefore quoted, are incorrectly split:
```auto
2025-10-16 20:54:44,"Generic Remote File/Path Include Attempt 4 (dir param, http/https)",4,64.39.106.3,10.97.219.194,/Common/pdc-www-pr_80,Remote File Include,1.0
```
the second field, `Signature_Name`, is terminated at the comma: `"Signature_Name"=>"\"Generic Remote File/Path Include Attempt 4 (dir param",`, and a new autonamed field gets added: `"column9"=>"1.0"`
The whole record, then, get rejected by Elastic:
`Could not index event to Elasticsearch. {:status=>400, [. . .] "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [Source_Ip] of type [ip] in document with id 'qFaL8ZkBMYlRvXlrtBPW'. Preview of field's value: '4'" `
The plugin version is `logstash-filter-csv (3.1.1)`. As it was released in June 2021, I suppose this bug isn't going to be fixed, so any advice for a workaround is welcome
Paolo
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 17, 2025, 1:10pm UTC](https://discuss.elastic.co/t/problem-with-quotes-in-csv-filter/382802/2 "2025-10-17T13:10:43Z")
> [@parosio](#):
>
> The plugin version is `logstash-filter-csv (3.1.1)`. As it was released in June 2021, I suppose this bug isn't going to be fixed, so any advice for a workaround is welcome
I don’t think most folks think it is a bug. If a field contains a comma it has to be quoted with “, and if quoted then the entire field must be quoted and any quotes within the field must be changed to ““. That’s just the way Ruby works.
If you have a recognizable pattern like a pair of IP addresses you could try parsing the line using grok.
---
### Author:  [@RainTown](https://discuss.elastic.co/u/RainTown)
#### Post date: [October 17, 2025, 3:14pm UTC](https://discuss.elastic.co/t/problem-with-quotes-in-csv-filter/382802/3 "2025-10-17T15:14:34Z")
There’s various tools like csvlint to check CSV validity, against eg RFC-4180.
I think we’ve all struggled with this type of thing before. My usual approach is to try to find a separator that just wont be in any field, as you generally know a bit about the expected data. And then double check with simple tools via awk or similar.
---
### Author:  [@parosio](https://discuss.elastic.co/u/parosio)
#### Post date: [October 17, 2025, 6:00pm UTC](https://discuss.elastic.co/t/problem-with-quotes-in-csv-filter/382802/4 "2025-10-17T18:00:47Z")
My problem is not with the fields quoted because they contain commas, but with those that contain quoted strings (i.e. `...This should be a "single" field...` ), like the one in the first example.
But I see now it's not rfc-4180 compliant: "_If fields are not enclosed with double quotes, then double quotes may not appear inside the fields._"
The "problematic" field is only the second one, so I suppose I could use some combination of dissect and grok as a workaround, but I hoped in something simpler.
PS definitely not a ruby CSV bug
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 17, 2025, 6:33pm UTC](https://discuss.elastic.co/t/problem-with-quotes-in-csv-filter/382802/5 "2025-10-17T18:33:40Z")
What is the source of this data? Do you have any control on how this data is formated?
---
### Author:  [@parosio](https://discuss.elastic.co/u/parosio)
#### Post date: [October 18, 2025, 3:13pm UTC](https://discuss.elastic.co/t/problem-with-quotes-in-csv-filter/382802/6 "2025-10-18T15:13:22Z")
It's an output from an IBM QRadar AQL Saved Search. The endpoint is documentd [here](https://ibmsecuritydocs.github.io/qradar_api_20.0/20.0--ariel-searches-search_id-results-GET.html) (funny it says _[...] The **formats are RFC compliant** and can be JSON, CSV, XML, or tabular text._ )
I thought requesting application/csv as return format would've saved me the json=\>csv conversion (csv is more concise).
However, yes, I could switch to json output (and then the json filter), or edit the saved search and move the offending field in last position (and then use dissect)
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 18, 2025, 3:39pm UTC](https://discuss.elastic.co/t/problem-with-quotes-in-csv-filter/382802/7 "2025-10-18T15:39:46Z")
If you can change to JSON I would recommend that, it is way more easier to work on Logstash.