# Extract the value from CSV field and add new field

**URL:** https://discuss.elastic.co/t/extract-the-value-from-csv-field-and-add-new-field/89974
**Category:** Logstash
**Created:** [June 19, 2017, 5:04pm UTC](https://discuss.elastic.co/t/extract-the-value-from-csv-field-and-add-new-field/89974 "2017-06-19T17:04:48Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![mussa572](https://avatars.discourse-cdn.com/v4/letter/m/a698b9/32.png) [@mussa572](https://discuss.elastic.co/u/mussa572)
#### Post date: [June 19, 2017, 5:04pm UTC](https://discuss.elastic.co/t/extract-the-value-from-csv-field-and-add-new-field/89974/1 "2017-06-19T17:04:48Z")

</div>

Dear Team

Wondering if you can help please . We are parsing the CSV using the below code , One of the field (exec) in CSV file can have either one of below values

(T= some integer value |D= some integer value |N=some integer value);

I want to be able to first check which value was add on the field . For example if its (T=5462) then I want logstash to create a new field called Throttling and send the value to elastic search . So elasticserch should see the Throttling as 5462 . If the value is N then should add new field "Network" orif its D then "Disk" and their respective values .

T= Throttling  
N = Network  
D= Disk

* * *

input {  
beats {  
port =\> "5044"  
}

}  
filter {  
csv {

```
  columns => ["time","exec","latency","orderid","src_ip","src_port","userid","obid","conn","msgtype"]
  separator => ","
  skip_empty_columns => "true"

```

}  
if[msgtype]=="D" {  
mutate {  
replace =\> ["msgtype","Single"]  
}  
}  
if[msgtype]=="G" {  
mutate {  
replace=\>["msgtype","Order Can"]  
}  
}  
if[msgtype]=="F" {  
mutate {  
replace=\>["msgtype","Order Can req"]  
}  
}

}

output {

elasticsearch {  
hosts =\> ["127.0.0.1:9200"]  
index =\> "logs"  
}  
stdout {}  
}

Regards

Mussa

---

<div class="post-metadata">

### Author: ![mussa572](https://avatars.discourse-cdn.com/v4/letter/m/a698b9/32.png) [@mussa572](https://discuss.elastic.co/u/mussa572)
#### Post date: [June 20, 2017, 12:34pm UTC](https://discuss.elastic.co/t/extract-the-value-from-csv-field-and-add-new-field/89974/2 "2017-06-20T12:34:03Z")

</div>

wondering if i can get any help on above please

---

<div class="post-metadata">

### Author: ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)
#### Post date: [June 20, 2017, 1:26pm UTC](https://discuss.elastic.co/t/extract-the-value-from-csv-field-and-add-new-field/89974/3 "2017-06-20T13:26:55Z")

</div>

you can use a grok pattern to extract both sides of the "" in that field with

```auto
filter {
  grok { match => ["msgtype", "%{WORD:msg_type_letter}=%{NUMBER:msg_type_value}" }
}

```

then you can use "if/else if/else if" instead of just "if/if/if":

```auto
if [msg_type_letter] == "T" {
   mutate { rename => { "msg_type_value" => "Throttling" } }
} else if [msg_type_letter] == "D" {
   mutate { rename => { "msg_type_value" => "Disk" } }
} else if [msg_type_letter] == "N" {
   mutate { rename => { "msg_type_value" => "Network" } }
}

```

---

<div class="post-metadata">

### Author: ![mussa572](https://avatars.discourse-cdn.com/v4/letter/m/a698b9/32.png) [@mussa572](https://discuss.elastic.co/u/mussa572)
#### Post date: [June 20, 2017, 5:17pm UTC](https://discuss.elastic.co/t/extract-the-value-from-csv-field-and-add-new-field/89974/4 "2017-06-20T17:17:36Z")

</div>

Hi jsvd

Thanks for above info manage to get the value with some changes to above code

grok {

match =\> ["excuses", "%{WORD:excuses\_type\_letter}=%{NUMBER:excuses\_type\_value}"]

}

if [excuses\_type\_letter] == "T" {

mutate { add\_field =\> { "Throttling" =\> "%{excuses\_type\_value}"} }

}

Now seeing one more issue , Logstash is sending all 3 fields value to elasticsearch instead I only want Throttling value to be send to Elasticsearch and ignore all other values .

What code would I need to stop Logstash from sending excuses , excuses\_type\_letter and excuses\_type\_value?

* * *

 ![](https://us1.discourse-cdn.com/elastic/original/3X/3/3/332bee87ffef4f242515c91424069db8465723e1.png)

---

<div class="post-metadata">

### Author: ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)
#### Post date: [June 21, 2017, 7:57am UTC](https://discuss.elastic.co/t/extract-the-value-from-csv-field-and-add-new-field/89974/5 "2017-06-21T07:57:08Z")

</div>

u can use

> if [excuses\_type\_letter] == "T" {

> mutate {  
> add\_field =\> { "Throttling" =\> "%{excuses\_type\_value}"}  
> remove\_field =\> ["%{excuses\_type\_letter}", "%{excuses\_type\_value}"]  
> }  
> }

[https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-remove\_field](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-remove_field)

---

<div class="post-metadata">

### Author: ![mussa572](https://avatars.discourse-cdn.com/v4/letter/m/a698b9/32.png) [@mussa572](https://discuss.elastic.co/u/mussa572)
#### Post date: [June 21, 2017, 9:54am UTC](https://discuss.elastic.co/t/extract-the-value-from-csv-field-and-add-new-field/89974/6 "2017-06-21T09:54:25Z")

</div>

Hi Tatdat

Thanks for the reply , i have added the code , however Logstash still sending excuses\_type\_letter and {excuses\_type\_value values to elastisearch .

Regards

Mussa

---

<div class="post-metadata">

### Author: ![mussa572](https://avatars.discourse-cdn.com/v4/letter/m/a698b9/32.png) [@mussa572](https://discuss.elastic.co/u/mussa572)
#### Post date: [June 21, 2017, 10:57am UTC](https://discuss.elastic.co/t/extract-the-value-from-csv-field-and-add-new-field/89974/7 "2017-06-21T10:57:34Z")

</div>

Hi

It worked after changing to following code . Thanks for the assistance .

mutate {

remove\_field =\> ["[excuses\_type\_letter]" ]  
remove\_field =\> ["[excuses\_type\_value]" ]  
}

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 19, 2017, 10:57am UTC](https://discuss.elastic.co/t/extract-the-value-from-csv-field-and-add-new-field/89974/8 "2017-07-19T10:57:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
