# Variable length csv

**URL:** <https://discuss.elastic.co/t/variable-length-csv/99867>\
**Category:** Logstash\
**Created:** [September 8, 2017, 2:43pm UTC](https://discuss.elastic.co/t/variable-length-csv/99867 "2017-09-08T14:43:41Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Arun\_Kaliraja\_Baskar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arun_kaliraja_baskar/32/21921_2.png) [@Arun\_Kaliraja\_Baskar](https://discuss.elastic.co/u/Arun_Kaliraja_Baskar)\
**Post date:** [September 8, 2017, 2:43pm UTC](https://discuss.elastic.co/t/variable-length-csv/99867/1 "2017-09-08T14:43:41Z")

</div>

Hi,

I have a csv file where data is organised in the may formats:

## format1:

value1, value2, value3,value4

## format2:

value1, value2, value3, keyX:valueX,keyY:valueY,keyZ:valueZ..

in the 2nd format my key:value will be of variable length..

How to write filters that can accomodate these two ?

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [September 8, 2017, 6:58pm UTC](https://discuss.elastic.co/t/variable-length-csv/99867/2 "2017-09-08T18:58:42Z")

</div>

From your formats listed, I assume that the first 3 fields are definitely CSV and the fourth field is either a value or a KV set.

You have a few options.

1. Use the CSV filter, defining 4 columns. Then in an if block test whether the fourth field contains a `:` colon (presuming that a non KV type fourth field does cannot have a colon in it). Then in the if block use the KV filter to split the keys and values on the colon.
2. If the first three field's values can never have a `,` comma in them (even in quotes), use the Dissect filter to split the 4 fields. Then use the if block and KV filter as above.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 6, 2017, 7:21pm UTC](https://discuss.elastic.co/t/variable-length-csv/99867/3 "2017-10-06T19:21:21Z")

</div>

Closed on October 6, 2017, 7:21pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.