# CSV quote char inside data

**URL:** https://discuss.elastic.co/t/csv-quote-char-inside-data/289290
**Category:** Logstash
**Created:** [November 16, 2021, 9:47am UTC](https://discuss.elastic.co/t/csv-quote-char-inside-data/289290 "2021-11-16T09:47:59Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![nosql\_injection](https://avatars.discourse-cdn.com/v4/letter/n/4491bb/32.png) [@nosql\_injection](https://discuss.elastic.co/u/nosql_injection)
#### Post date: [November 16, 2021, 9:47am UTC](https://discuss.elastic.co/t/csv-quote-char-inside-data/289290/1 "2021-11-16T09:47:59Z")

</div>

Hi there,

I'm trying to ingest syslog data with the CSV filter.

I guess the issue is related to the " inside the URL part ([acme.org](http://acme.org)&iid={ **"** -123456\*\*"\*\*:4}&sid=123458&tid=123) of the log, which is the same char as the "quote\_char".

Is there an elegant solution to this? (I can't change the input format ;-))

Many thanks!  
Markus

Logstash Error

```auto
[2021-11-16T07:00:10,105][WARN][logstash.filters.csv] Error parsing csv {:field=>"message", :source=>"\"Tue Nov 12 12:12:12 2021\",\"xxx\",\"HTTP\",\"acme.org&iid={\"-123456\":4}&sid=123458&tid=123\",\"Allowed\",\"General Browsing\",\"General Browsing\",\"1332\",\"432\",\"188\",\"188\",\"Business Use\",\"Information Technology\",\"Web Search\",\"None\",\"None\",\"0\",\"None\",\"None\",\"ABC\",\"R&D\",\"192.168.1.2\",\"1.2.3.4\",\"GET\",\"200\",\"ABC\",\"None\",\"None\",\"None\",\"image/gif\",\"None\",\"123\",\"123\"\n", :exception=>#<CSV::MalformedCSVError: Missing or stray quote in line 1>}

```

Logstash Pipeline

```auto
input {
  syslog {
    port => 1234
    tags => ["some-logs"]
  }
}

filter {
        if "some-logs" in [tags] {
                csv {
                     columns => ["time","login","proto","eurl","action","appname","appclass","reqsize","respsize","stime","ctime","urlclass","urlsupercat","urlcat","malwarecat","threatname","riskscore","dlpeng","dlpdict","location","dept","cip","sip","reqmethod","respcode","ua","ereferer","ruletype","rulelabel","contenttype","unscannabletype","deviceowner","devicehostname"]
                    }
        }
}

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 16, 2021, 5:44pm UTC](https://discuss.elastic.co/t/csv-quote-char-inside-data/289290/2 "2021-11-16T17:44:38Z")

</div>

In a CSV if a field contains double quotes then the entire field must be enclosed in double quotes, and any double quotes within the field must be escaped with a second double quote (see items 5, 6, and 7 in [section 2](https://datatracker.ietf.org/doc/html/rfc4180#page-2) of RFC 4180) --

```
 foo,"a ""b"" c",bar

```

The underlying Ruby CSV class has a liberal\_parsing option that relaxes these requirements, but the csv filter does not set or expose it.

You will need to modify the message format. You may be able to do that using mutate+gsub if the input format is predictable enough.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 14, 2021, 5:45pm UTC](https://discuss.elastic.co/t/csv-quote-char-inside-data/289290/3 "2021-12-14T17:45:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
