# Splitting a column in a csv

**URL:** <https://discuss.elastic.co/t/splitting-a-column-in-a-csv/72043>\
**Category:** Logstash\
**Created:** [January 18, 2017, 3:59pm UTC](https://discuss.elastic.co/t/splitting-a-column-in-a-csv/72043 "2017-01-18T15:59:00Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pokecallum](https://avatars.discourse-cdn.com/v4/letter/p/d6d6ee/32.png) [@Pokecallum](https://discuss.elastic.co/u/Pokecallum)\
**Post date:** [January 18, 2017, 3:59pm UTC](https://discuss.elastic.co/t/splitting-a-column-in-a-csv/72043/1 "2017-01-18T15:59:00Z")

</div>

I have a csv file simular to this

Date|ID|Recipient|Subject  
1-01-2016|123|test@test.com|A test  
2-01-2016|124|test@test.com, test123@test.com|another test

Sorry if this gets confusing, I have used | to show where each column is defined in my csv file But it is delimited with a ,

On my second line, there are two recipients is there anyway to either delete the second recipient before it is indexed or give it its own column

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 20, 2017, 6:51am UTC](https://discuss.elastic.co/t/splitting-a-column-in-a-csv/72043/2 "2017-01-20T06:51:05Z")

</div>

Sure, that's possible. You can process the extracted third field with grok, mutate, or any other filter put after the csv filter.

---

<div class="post-metadata">

**Author:** ![Pokecallum](https://avatars.discourse-cdn.com/v4/letter/p/d6d6ee/32.png) [@Pokecallum](https://discuss.elastic.co/u/Pokecallum)\
**Post date:** [January 20, 2017, 9:31am UTC](https://discuss.elastic.co/t/splitting-a-column-in-a-csv/72043/3 "2017-01-20T09:31:29Z")

</div>

Would i be able to count how many times a domain appears in the field ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 20, 2017, 9:48am UTC](https://discuss.elastic.co/t/splitting-a-column-in-a-csv/72043/4 "2017-01-20T09:48:59Z")

</div>

With a ruby filter you can do pretty much what you want.

---

<div class="post-metadata">

**Author:** ![Pokecallum](https://avatars.discourse-cdn.com/v4/letter/p/d6d6ee/32.png) [@Pokecallum](https://discuss.elastic.co/u/Pokecallum)\
**Post date:** [January 23, 2017, 5:13pm UTC](https://discuss.elastic.co/t/splitting-a-column-in-a-csv/72043/5 "2017-01-23T17:13:05Z")

</div>

I've created a ruby scipt thatnow does what I want, Its basically just a bunch of counts, how do i go about adding the variables to the end of each line. So that they are inputed as well

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 23, 2017, 9:22pm UTC](https://discuss.elastic.co/t/splitting-a-column-in-a-csv/72043/6 "2017-01-23T21:22:42Z")

</div>

To the end of each line? What output plugin(s) are you using? What do your processed events currently look like? Use a `stdout { codec => rubydebug }` output to see exactly what you've got.

---

<div class="post-metadata">

**Author:** ![Pokecallum](https://avatars.discourse-cdn.com/v4/letter/p/d6d6ee/32.png) [@Pokecallum](https://discuss.elastic.co/u/Pokecallum)\
**Post date:** [January 24, 2017, 10:50am UTC](https://discuss.elastic.co/t/splitting-a-column-in-a-csv/72043/7 "2017-01-24T10:50:46Z")

</div>

I know the add field =\> doesnt work..

What i want to do is the input into elasticsearch will have the following fields

Date|ID|Recipient|Subject|Internal Count|Home Use Count|Buisness Partners Count|Managed Suppliers| All|Unknown

The last 6 getting there counts from the ruby variables.

Sorry orginally this was being inputted with the csv filter so using ruby is really new to me

```
{
		'path' => "/home/callum/test/raw.csv",
		@timestamp = > 2017-01-24T10:34:31,
		@version => "1",
		"host" => "ubuntu"
		"message" => "ZIP,2017-01-10 08:53 GMT,48784048,myhost (10.15.193.196),This-is-the-sender@gmail.com,reciver@gmail.com,anotherreciver@yahoo.com,andanother@gamail.com,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Email Subject\r",
		"tags" => {
		[0] "_rubyexception"
		]
	}

```

 ![](https://us1.discourse-cdn.com/elastic/original/2X/e/e74f61fb4c32d006d76ff9cc97f2d558fa0d39d6.PNG)

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 24, 2017, 11:00am UTC](https://discuss.elastic.co/t/splitting-a-column-in-a-csv/72043/8 "2017-01-24T11:00:11Z")

</div>

Your Ruby code is broken so you're getting a `_rubyexception` tag. Why? Look in the Logstash log for clues.

Taking a quick look at the code it seems like you're calling a lot of methods in the event object that don't exist.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 21, 2017, 11:00am UTC](https://discuss.elastic.co/t/splitting-a-column-in-a-csv/72043/9 "2017-02-21T11:00:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
