# On ingest I need to create a new field based on the value of a field in the document

**URL:** https://discuss.elastic.co/t/on-ingest-i-need-to-create-a-new-field-based-on-the-value-of-a-field-in-the-document/64041
**Category:** Logstash
**Created:** [October 26, 2016, 5:51pm UTC](https://discuss.elastic.co/t/on-ingest-i-need-to-create-a-new-field-based-on-the-value-of-a-field-in-the-document/64041 "2016-10-26T17:51:14Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Christopher\_Pinto](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christopher_pinto/32/12746_2.png) [@Christopher\_Pinto](https://discuss.elastic.co/u/Christopher_Pinto)
#### Post date: [October 26, 2016, 5:51pm UTC](https://discuss.elastic.co/t/on-ingest-i-need-to-create-a-new-field-based-on-the-value-of-a-field-in-the-document/64041/1 "2016-10-26T17:51:14Z")

</div>

I have a field X that has the value fw.auth.deny or fw.auth.allow How can I create a new field that is called y that has the value fw in it using logstash or mapping templates?

Thanks in advance.

Chris

P.S. I looked around a lot and could not find an example of how to do this.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 26, 2016, 8:12pm UTC](https://discuss.elastic.co/t/on-ingest-i-need-to-create-a-new-field-based-on-the-value-of-a-field-in-the-document/64041/2 "2016-10-26T20:12:04Z")

</div>

Use a grok filter. Why should "fw" be extracted? Does X always begin with "fw"? Or do you want to extract everything up to the first period?

---

<div class="post-metadata">

### Author: ![Christopher\_Pinto](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christopher_pinto/32/12746_2.png) [@Christopher\_Pinto](https://discuss.elastic.co/u/Christopher_Pinto)
#### Post date: [November 1, 2016, 4:43pm UTC](https://discuss.elastic.co/t/on-ingest-i-need-to-create-a-new-field-based-on-the-value-of-a-field-in-the-document/64041/3 "2016-11-01T16:43:17Z")

</div>

I have a bunch of patterns that I need to match

fw.auth.allow  
fw.auth.deny  
should set a new field to fw

av.detect.virus  
av.delete.virus  
av.detect.content  
av.delete.content  
should set a new field to av

and etc, etc, etc

Is there some where I can find a logstash config file with a similar example to what I need to do?

Chris  
P.S. Thanks.

---

<div class="post-metadata">

### Author: ![Christopher\_Pinto](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christopher_pinto/32/12746_2.png) [@Christopher\_Pinto](https://discuss.elastic.co/u/Christopher_Pinto)
#### Post date: [November 1, 2016, 6:29pm UTC](https://discuss.elastic.co/t/on-ingest-i-need-to-create-a-new-field-based-on-the-value-of-a-field-in-the-document/64041/4 "2016-11-01T18:29:18Z")

</div>

Would this work?

input {  
file {  
path =\> "/data2/logstash/\*.csv"  
type =\> "utm"  
start\_position =\> "beginning"  
}  
}

filter {  
csv {  
columns =\> ["g\_date", "g\_hour", "g\_hostname", "nsm\_type",  
"s\_ip", "t\_ip", "n\_ip", "t\_port", "transport\_proto", "in\_interface",  
"out\_interface", "group\_id", "user\_id", "object\_name", "object\_access",  
"http\_hostname", "uri", "s\_port", "s\_bytes", "t\_bytes", "duration",  
"g\_timestamp"]  
separator =\> ","  
}  
mutate {  
lowercase =\> ["g\_hostname"]  
}  
}

filter {  
if [nsm\_type] == "fw.auth.allow" [nsm\_type] == "fw.auth.deny" {  
grok {  
add\_field =\> ["service", "fw"]  
}  
}  
}

output {  
elasticsearch {  
action =\> "index"  
hosts =\> ["10.140.56.140:9700", "10.140.56.141:9700", "10.140.56.142:9700", "10.140.56.143:9700", "10.140.56.144:9700", "10.140.56.145:9700"]  
index =\> "utm-%{g\_date}"  
workers =\> 12  
}

# stdout {

# codec =\> rubydebug

# }

}

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [November 1, 2016, 6:39pm UTC](https://discuss.elastic.co/t/on-ingest-i-need-to-create-a-new-field-based-on-the-value-of-a-field-in-the-document/64041/5 "2016-11-01T18:39:24Z")

</div>

Looks pretty reasonable. To obtain the first period-delimited token from `nsm_type`, use a grok filter that extracts all characters at the beginning of the string except periods into a new field:

```nohighlight
grok {
  match => {
    "nsm_type" => "^(?<desired-name-of-field>[^.]+)"
  }
}

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:31am UTC](https://discuss.elastic.co/t/on-ingest-i-need-to-create-a-new-field-based-on-the-value-of-a-field-in-the-document/64041/6 "2017-07-06T04:31:42Z")

</div>


