# Parsing csv file which has some field/column containing separator as values

**URL:** https://discuss.elastic.co/t/parsing-csv-file-which-has-some-field-column-containing-separator-as-values/60768
**Category:** Logstash
**Created:** [September 17, 2016, 9:01am UTC](https://discuss.elastic.co/t/parsing-csv-file-which-has-some-field-column-containing-separator-as-values/60768 "2016-09-17T09:01:05Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Immortal](https://avatars.discourse-cdn.com/v4/letter/i/f08c70/32.png) [@Immortal](https://discuss.elastic.co/u/Immortal)
#### Post date: [September 17, 2016, 9:01am UTC](https://discuss.elastic.co/t/parsing-csv-file-which-has-some-field-column-containing-separator-as-values/60768/1 "2016-09-17T09:01:05Z")

</div>

I have following type of entries in csv file:

16 , 5 , 53 , Logging/LoggingError , 2, error while opening file "a,b,c,d": , 10.10.4.219 , serviceA

I am trying to parse it using csv pluging as follows:

csv {

```
columns => ["time","split",event","event_name","level","error_msg","ip","service"]
separator => ","

```

}

Now problem is i have used ","(comma) as separator but in my error\_msg column there are some "," present so logstash is parsing the values thinking as separator. In actual log file there are different error\_msg with different number of ","(commas).

So please help me how i can solve this problem!!

Thanks in advance 🙂

---

<div class="post-metadata">

### Author: ![nellicus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nellicus/32/51566_2.png) [@nellicus](https://discuss.elastic.co/u/nellicus)
#### Post date: [September 17, 2016, 12:25pm UTC](https://discuss.elastic.co/t/parsing-csv-file-which-has-some-field-column-containing-separator-as-values/60768/2 "2016-09-17T12:25:04Z")

</div>

> [@Immortal](#):
>
> in my error\_msg column there are some "," present

can you show the _actual_ raw event as it is in the log file you're trying to parse?

perhaps [mutate=\>gsub](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-gsub) could help by tidying up the event before sending it to csv filter....

---

<div class="post-metadata">

### Author: ![bhatch](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@bhatch](https://discuss.elastic.co/u/bhatch)
#### Post date: [September 19, 2016, 11:02pm UTC](https://discuss.elastic.co/t/parsing-csv-file-which-has-some-field-column-containing-separator-as-values/60768/3 "2016-09-19T23:02:20Z")

</div>

That line of data isn't properly formatted for a csv.

You have an unescaped comma and unescaped double quote in the text. So if you use a text qualifier, it will fail because of the unescaped double quote. If you don't use a text qualifier, it will fail on the unescaped comma. There is no real solution without changing the data.

Ideally that line should look like:

`16 , 5 , 53 , "Logging/LoggingError" , 2, "error while opening file ""a,b,c,d"":" , "10.10.4.219" , "serviceA"`

By doubling up the double quotes you are telling the CSV parser to escape it. But if you can't change that source data you may be out of luck.

---

<div class="post-metadata">

### Author: ![Immortal](https://avatars.discourse-cdn.com/v4/letter/i/f08c70/32.png) [@Immortal](https://discuss.elastic.co/u/Immortal)
#### Post date: [September 20, 2016, 6:02am UTC](https://discuss.elastic.co/t/parsing-csv-file-which-has-some-field-column-containing-separator-as-values/60768/4 "2016-09-20T06:02:59Z")

</div>

Thank u all for suggestions.

I used mutate =\>gsub to replace the ","(comma) present inside the column with ""(blank).  
Now its parsing correctly for single event using:  
input {  
stdin {  
type =\> "application1"}  
}

But for reading complete file the fields are distributed across multiline .....  
So how can i use multiline pattern for a csv file.

I could not find anything for multiline csv files!!!  
Please suggest some solution for multiline csv file

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:37am UTC](https://discuss.elastic.co/t/parsing-csv-file-which-has-some-field-column-containing-separator-as-values/60768/5 "2017-07-06T04:37:53Z")

</div>


