# Parse csv with variable number of columns

**URL:** <https://discuss.elastic.co/t/parse-csv-with-variable-number-of-columns/165170>\
**Category:** Logstash\
**Created:** [January 22, 2019, 7:58am UTC](https://discuss.elastic.co/t/parse-csv-with-variable-number-of-columns/165170 "2019-01-22T07:58:58Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Petr.Simik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/petr.simik/32/38082_2.png) [@Petr.Simik](https://discuss.elastic.co/u/Petr.Simik)\
**Post date:** [January 22, 2019, 7:58am UTC](https://discuss.elastic.co/t/parse-csv-with-variable-number-of-columns/165170/1 "2019-01-22T07:58:58Z")

</div>

What would be the best way to parse simple file  
with the structure where second column "optional\_retry\_count" sometime exist sometime not.

> message\_type, optional\_retry\_count, date, xml\_data
> 
> EURec,20190121150448,\<?xml version="1.0"....  
> EUSlc,2,20190121150458,\<?xml version="1.0"....  
> EUNot,3,20190121150468,\<?xml version="1.0"....  
> EUSet,20190121150488,\<?xml version="1.0"....  
> EURec,0,20190121150498,\<?xml version="1.0"....

using:

> filter {  
> grok {  
> match =\> { "message" =\> "%{DATA:type},%{DATA:date},%{GREEDYDATA:xmldata}" }  
> }

Which is the bests way to decode optional 2nd field Retry

> %{DATA:retry}

should I use Grok, Dissect, csv and how to incorporate the condition?

expected result:

> {  
> "date": "20190121150458",  
> "type": "EUSlc",  
> "xmldata": "\<?xml version="1.0"....\r",  
> "retry": "2"  
> }

Thank you

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 22, 2019, 1:18pm UTC](https://discuss.elastic.co/t/parse-csv-with-variable-number-of-columns/165170/2 "2019-01-22T13:18:57Z")

</div>

A csv filter will not like the XML data, so I would use dissect.

```
if [message] =~ /^[^,]+,[^,]+,[^,]+,<\?xml/ {
    dissect { mapping => { "message" => "%{message_type},%{retry_count},%{date},%{xml_data}" } }
} else if [message] =~ /^[^,]+,[^,]+,<\?xml/ {
    dissect { mapping => { "message" => "%{message_type},%{date},%{xml_data}" } }
} else {
    mutate { add_tag => ["somethingElse"] }
}
```

---

<div class="post-metadata">

**Author:** ![Petr.Simik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/petr.simik/32/38082_2.png) [@Petr.Simik](https://discuss.elastic.co/u/Petr.Simik)\
**Post date:** [January 22, 2019, 1:53pm UTC](https://discuss.elastic.co/t/parse-csv-with-variable-number-of-columns/165170/3 "2019-01-22T13:53:43Z")

</div>

Thank you this solved my problem.  
excellent fast answer

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 19, 2019, 1:53pm UTC](https://discuss.elastic.co/t/parse-csv-with-variable-number-of-columns/165170/4 "2019-02-19T13:53:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.