# Special occurences

**URL:** https://discuss.elastic.co/t/special-occurences/142652
**Category:** Logstash
**Created:** [August 1, 2018, 8:57pm UTC](https://discuss.elastic.co/t/special-occurences/142652 "2018-08-01T20:57:58Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![dersoi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dersoi/32/32916_2.png) [@dersoi](https://discuss.elastic.co/u/dersoi)
#### Post date: [August 1, 2018, 8:57pm UTC](https://discuss.elastic.co/t/special-occurences/142652/1 "2018-08-01T20:57:59Z")

</div>

Hi,

I use my ELK stack stock logs, and sometimes, logs will be in this format  
field1, field2, field3,  
and so on, so I basically CSV to separate the fields. A little problem rose... Sometimes, the field will actually be like this:  
field1, "fie, ld2", field 3

The way I thought of to solve this issue would be to count the number of commas: if there are more than the expected number, it means that field2 contains one, and therefore, I should use the quotes to see whats in there...  
However, I could not find a way to do that in Logstash.

Does somebody have an idea how to count the commas, or maybe a better way to proceed?

Thanks,  
Matthias

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 1, 2018, 9:36pm UTC](https://discuss.elastic.co/t/special-occurences/142652/2 "2018-08-01T21:36:44Z")

</div>

> [@dersoi](#):
>
> field1, "fie, ld2", field 3

A csv filter will handle quoted fields, but the entire field has to be quoted, with no leading space. So this might work, but it is going to be fragile.

```
    mutate { gsub => ["message", ', "', ',"'] }
    csv { source => "message" }

```

---

<div class="post-metadata">

### Author: ![dersoi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dersoi/32/32916_2.png) [@dersoi](https://discuss.elastic.co/u/dersoi)
#### Post date: [August 1, 2018, 9:57pm UTC](https://discuss.elastic.co/t/special-occurences/142652/3 "2018-08-01T21:57:05Z")

</div>

Hi Badger,

If I'm not mistaken, your solution supposes that every single log is written

field1, "fie, ld2", field3

As I have stated it is not always the case.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 1, 2018, 10:52pm UTC](https://discuss.elastic.co/t/special-occurences/142652/4 "2018-08-01T22:52:49Z")

</div>

> [@dersoi](#):
>
> As I have stated it is not always the case.

I do not believe I have made that supposition.

---

<div class="post-metadata">

### Author: ![dersoi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dersoi/32/32916_2.png) [@dersoi](https://discuss.elastic.co/u/dersoi)
#### Post date: [August 2, 2018, 2:10pm UTC](https://discuss.elastic.co/t/special-occurences/142652/5 "2018-08-02T14:10:18Z")

</div>

Oh yes, my bad, I misunderstood your code. If (now) understand correctly, your code will delete any comma that is between quotes, right?

If that is the case, although I thank you for this idea, is there no way to conserve the comma? I would prefer not altering the logs...

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 2, 2018, 2:15pm UTC](https://discuss.elastic.co/t/special-occurences/142652/6 "2018-08-02T14:15:41Z")

</div>

No, the gsub remove a single space between a comma and a double quote.

If you really need to preserve the exact format of the logs you'll could fork the [csv filter code](https://github.com/logstash-plugins/logstash-filter-csv) and rewrite it to handle partially quoted fields.

---

<div class="post-metadata">

### Author: ![dersoi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dersoi/32/32916_2.png) [@dersoi](https://discuss.elastic.co/u/dersoi)
#### Post date: [August 2, 2018, 2:43pm UTC](https://discuss.elastic.co/t/special-occurences/142652/7 "2018-08-02T14:43:37Z")

</div>

Ah, I see!  
I'll try and see if it works. Thank you very much!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 30, 2018, 2:43pm UTC](https://discuss.elastic.co/t/special-occurences/142652/8 "2018-08-30T14:43:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
