# Split csv column to several fields

**URL:** <https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927>\
**Category:** Logstash\
**Created:** [December 10, 2015, 11:17pm UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927 "2015-12-10T23:17:52Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\
**Post date:** [December 10, 2015, 11:17pm UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/1 "2015-12-10T23:17:52Z")

</div>

i use logstash to import from csv to elastic. the problem is that one of the fields contains the string formatted as "int|string|string" and i need to parse this and put into nested field. what are possible solutions for this?  
Also this field is array

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 11, 2015, 7:18am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/2 "2015-12-11T07:18:29Z")

</div>

Sounds like the mutate filter's [`split` option](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-split) would be a good fit.

---

<div class="post-metadata">

**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\
**Post date:** [December 13, 2015, 11:25pm UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/3 "2015-12-13T23:25:55Z")

</div>

the first problem is that it is array of that strings divided by ", ". the second one is that array should be added as nested objects in elastic and the parts of that string are fields of nested object. i have no idea how i can use split for this.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 14, 2015, 6:36am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/4 "2015-12-14T06:36:41Z")

</div>

Sorry, I don't understand what you're talking about. Describing the problem with examples is usually better than using words. Show us what event you have and what you'd like to achieve.

---

<div class="post-metadata">

**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\
**Post date:** [December 14, 2015, 8:57am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/5 "2015-12-14T08:57:37Z")

</div>

this column in csv contains the folowing: 1|http://blahblahblah.com|blahblahblah,23|http://anotheraddres.com/|anothertext

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 14, 2015, 8:58am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/6 "2015-12-14T08:58:30Z")

</div>

Yes? And what's the expected result?

---

<div class="post-metadata">

**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\
**Post date:** [December 14, 2015, 9:08am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/7 "2015-12-14T09:08:24Z")

</div>

in es document there is nested field where i want to insert these values, like this:  
links  
properties:[  
{ "[link.id](http://link.id)":"1",  
"link.url":"[http://blahblahblah.com](http://blahblahblah.com)",  
"[link.name](http://link.name)":"blahblahblah"  
},  
{ "[link.id](http://link.id)":"23",  
"link.url":"[http://anotheraddress.com/](http://anotheraddress.com/)",  
"[link.name](http://link.name)":"anothertext"  
}]

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 14, 2015, 9:35am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/8 "2015-12-14T09:35:01Z")

</div>

So you want to update a document that already exists and add new JSON objects to an array? That's... not so easy.

---

<div class="post-metadata">

**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\
**Post date:** [December 14, 2015, 9:47am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/9 "2015-12-14T09:47:01Z")

</div>

No i want to create new document

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 14, 2015, 9:58am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/10 "2015-12-14T09:58:04Z")

</div>

Right, but you want each line of the CSV file to build upon each other and in the end result in a single document containing all URL/name pairs found in the CSV file?

Because an input file isn't necessarily read in one pass and potentially could be very large you'd typically implement this by having it create an initial document and update it for each line of the input file.

---

<div class="post-metadata">

**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\
**Post date:** [December 14, 2015, 9:59am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/11 "2015-12-14T09:59:06Z")

</div>

one line =\>one doc

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 14, 2015, 11:43am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/12 "2015-12-14T11:43:45Z")

</div>

Okay, now it's all clear. I think you'll have to involve a ruby filter. Untested but should be fairly close:

```auto
ruby {
  code => "
    event['links'] = event['message'].split('\\,').collect { |t|
      c = t.split '|'
      {
        'link_id' => c[0],
        'link_url' => c[1],
        'link_name' => c[2]
      }
    }
  "
}

```

Note that Elasticsearch no longer allows periods in field names so I replaced them with underscores. If you want each element of the array to contain a single object with three keys (id, url, name) that's of course doable too.

---

<div class="post-metadata">

**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\
**Post date:** [December 14, 2015, 12:04pm UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/13 "2015-12-14T12:04:33Z")

</div>

thank u very much i will try it later.

---

<div class="post-metadata">

**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\
**Post date:** [December 14, 2015, 12:05pm UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/14 "2015-12-14T12:05:56Z")

</div>

but will it create multiple links for single document?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 14, 2015, 12:14pm UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/15 "2015-12-14T12:14:17Z")

</div>

> but will it create multiple links for single document?

Yes. I realized that I missed one crucial part earlier. I've updated my previous post so that the list of JSON objects is stored in the `links` field.

---

<div class="post-metadata">

**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\
**Post date:** [December 15, 2015, 10:25am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/16 "2015-12-15T10:25:01Z")

</div>

thanks a lot. It works !

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:18am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927/17 "2017-07-06T05:18:23Z")

</div>


