# Move the location of newly added add\_field out of nested?

**URL:** <https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143>\
**Category:** Logstash\
**Created:** [August 29, 2016, 5:00am UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143 "2016-08-29T05:00:58Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Allie\_Yang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/allie_yang/32/16538_2.png) [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\
**Post date:** [August 29, 2016, 5:00am UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/1 "2016-08-29T05:00:58Z")

</div>

Hi I can access the nested field by [field][sub\_field], and add this as a new field. However, this new field is still nested, then it is no use to me. How can i change its position out of net pls?

Below is an example. I add\_field created the **purchase.source** , but it is still nested within **purchase** field.  
 ![](https://us1.discourse-cdn.com/elastic/original/2X/b/b5194cb5a4c7aab4a062ddae76acd0e21130fbea.png)

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 29, 2016, 5:39am UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/2 "2016-08-29T05:39:22Z")

</div>

Please don't post screenshots. Use copy/paste.

It's hard to understand what you want to accomplish. If you don't want the field to be nested and you create it with `add_field` why not just change the field name to _not_ be nested? Or do you want the field name to literally be `purchase.source`? That won't work since field names with dots aren't allowed.

---

<div class="post-metadata">

**Author:** ![Allie\_Yang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/allie_yang/32/16538_2.png) [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\
**Post date:** [August 29, 2016, 5:35pm UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/3 "2016-08-29T17:35:38Z")

</div>

Hi thanks for the prompt reply and sorry for the screen shot. It still doesn't work as below.

"@timestamp" =\> "2016-08-29T17:32:23.321Z",  
"path" =\> "/Users/yangyan/Desktop/log\_file/test.log",  
"host" =\> "yangyan-osx",  
"type" =\> "json",  
"timestamp" =\> "Jan 1 00:01:18",  
"logsource" =\> "eqx-astockweb1",  
"program" =\> "adobestock",  
"pid" =\> "59741",  
"app\_id" =\> "as",  
"geid" =\> "61dc639d38fb5dc98e6d65a5c5903d1e",  
"etid" =\> "purchase",  
"ev" =\> 1,  
"date" =\> "2015-12-31 23:01:18",  
"ip" =\> "10.1.8.37",  
"asui" =\> "84e48f408d265e2f8ef36f26f75bdbcb",  
"session\_id" =\> "2193804e13f5a640afb1a1b80613281f",  
"member\_id" =\> -1,  
"is\_buyer" =\> false,  
"url" =\> "[https://stock.adobe.com/Callback/JEM/Provisioning](https://stock.adobe.com/Callback/JEM/Provisioning)",  
"locale" =\> "en\_US",  
"purchase" =\> {  
"source" =\> [  
[0] "jem",  
[1] " **pSource**"  
],  
"type\_id" =\> 3,  
"sao" =\> "323C16AD55DFE19B0A744C37",  
"order\_number" =\> nil,  
"content\_id" =\> nil,  
"delegate\_guid" =\> nil,  
"sku" =\> "65260923"

AND here is how i do the _add\_\_field_ within filter plugin:  
mutate{  
add\_field =\> {  
"[purchase][source]" =\> "pSource"  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 29, 2016, 5:40pm UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/4 "2016-08-29T17:40:48Z")

</div>

Okay, this is what your event currently looks like and what configuration you have. What's the expected result then?

---

<div class="post-metadata">

**Author:** ![Allie\_Yang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/allie_yang/32/16538_2.png) [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\
**Post date:** [August 29, 2016, 5:53pm UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/5 "2016-08-29T17:53:18Z")

</div>

Hi my conf is below:  
input {  
file {  
path =\> "/Users/yangyan/Desktop/log\_file/test.log"  
start\_position =\> "beginning"  
type =\> "json"  
codec =\> json  
}  
}

filter {  
grok{  
match =\> { "message" =\> "%{SYSLOGBASE} %{GREEDYDATA:message}" }  
overwrite =\> ["message"]  
}

json {  
source =\> "message"  
}

mutate{  
add\_field =\> { "[purchase][source]" =\> "pSource" }  
}

prune {  
whitelist\_names =\> ["timestamp", "app\_id", "date", "member\_id", "locale", "pSource"]  
}  
}

output {  
elasticsearch { }  
stdout { codec =\> rubydebug }  
}

THE original log file is like this:  
Jan 1 00:01:18 eqx-astockweb1 adobestock[59741]: {" **app\_id**":"as","geid":"61dc639d38fb5dc98e6d65a5c5903d1e","etid":"purchase","ev":1," **date**":"2015-12-31 23:01:18","mt":1451602878.69,"ip":"10.1.8.37","asui":"84e48f408d265e2f8ef36f26f75bdbcb","session\_id":"2193804e13f5a640afb1a1b80613281f"," **member\_id**":-1,"is\_buyer":false,"url":"[https://stock.adobe.com/Callback/JEM/Provisioning","](https://stock.adobe.com/Callback/JEM/Provisioning%22,%22) **locale**":"en\_US"," **purchase**":{" **source**":"jem"," **type\_id**":3,"sao":"323C16AD55DFE19B0A744C37","order\_number":null,"content\_id":null,"delegate\_guid":null,"sku":"65260923"}}

THE goal is to parse out only the **bolded** fields. And if possible, convert date to _Date_ type. (currently everything is default to String, since they are parsed out of json)

Pls kindly let me know!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 29, 2016, 6:04pm UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/6 "2016-08-29T18:04:20Z")

</div>

Use the mutate filter's rename option to move `[purchase][source]` somewhere else. Then use the prune filter to delete all fields except the ones you specifically list.

---

<div class="post-metadata">

**Author:** ![Allie\_Yang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/allie_yang/32/16538_2.png) [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\
**Post date:** [August 29, 2016, 6:17pm UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/7 "2016-08-29T18:17:19Z")

</div>

Thanks a lot, i got it!!! I wouldn't have notice rename would achieve this!

is there a way i can convert the **date** field, which is parsed to be a string out of json, into _Date_ type pls, except for using complex ruby?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 29, 2016, 6:19pm UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/8 "2016-08-29T18:19:18Z")

</div>

Look at the date filter.

---

<div class="post-metadata">

**Author:** ![Allie\_Yang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/allie_yang/32/16538_2.png) [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\
**Post date:** [August 29, 2016, 7:03pm UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/9 "2016-08-29T19:03:58Z")

</div>

Cool, i got it!  
Lastly, is there anyway i can show my fields' data type In logstash, Kibana, ES?  
Now i can see in kibana, that **date** field is still String; @timestamp does turns to be Date type...should i name a new indice in kibana maybe?  
Thanks~!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 29, 2016, 8:24pm UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/10 "2016-08-29T20:24:58Z")

</div>

Your `date` field might be a string because the first document with a `date` field didn't contain a string that ES could parse as a date. It therefore became a string instead, and that's not going to change without reindexing.

But yes, it would be a good idea to stick to the defaults without trying to use `date` instead of `@timestamp` etc. Once you understand better how things work you can deviate.

---

<div class="post-metadata">

**Author:** ![Allie\_Yang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/allie_yang/32/16538_2.png) [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\
**Post date:** [August 30, 2016, 12:10am UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/11 "2016-08-30T00:10:32Z")

</div>

Thanks! Does this mean I could not easily convert **String** type to **Date** type without reindex, if i need to maintain the default @timestamp as well?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 30, 2016, 5:49am UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/12 "2016-08-30T05:49:06Z")

</div>

An index's field mappings can't be changed without reindexing.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:40am UTC](https://discuss.elastic.co/t/move-the-location-of-newly-added-add-field-out-of-nested/59143/13 "2017-07-06T04:40:59Z")

</div>


