# Logstash Elasticsearch template for doc\_values

**URL:** https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423
**Category:** Logstash
**Created:** [September 30, 2015, 3:21pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423 "2015-09-30T15:21:46Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![elvarb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elvarb/32/44840_2.png) [@elvarb](https://discuss.elastic.co/u/elvarb)
#### Post date: [September 30, 2015, 3:21pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423/1 "2015-09-30T15:21:46Z")

</div>

How do I go about modifying the default template for Logstash for it to use doc\_values for as many fields as possible?

With Elasticsearch 2.0 doc\_values will be default for all fields instead of optional, so by then this problem should solve itself. Does this mean that I could instead of changing the Logstash template I could change the default in Elasticsearch?

I have found these discussions on the subject, but I'm not sure what is the right method.

> [@How do I add doc\_values to logstash index mapping template in Elasticsearch?](https://discuss.elastic.co/t/how-do-i-add-doc-values-to-logstash-index-mapping-template-in-elasticsearch/25385):
>
> This is the default logstash index mapping template I currently have in elasticsearch : { "order" : 0, "template" : "logstash-", "settings" : { "index.refresh\_interval" : "5s" }, "mappings" : { "default" : { "dynamic\_templates" : [ { "message\_field" : { "mapping" : { "index" : "analyzed", "omit\_norms" : true, "type" : "string" }, "match\_mapping\_type" : "string", "match" : "message" } }, { "string\_fields" : { "mapping" : { "index" : "analyzed", "omit\_norms" : true, "type…

> [@Is this an OK addition of doc\_values : true in elasticsearch template with logstash?](https://discuss.elastic.co/t/is-this-an-ok-addition-of-doc-values-true-in-elasticsearch-template-with-logstash/29361):
>
> Hi, got a little tired of memory issues and figured I'd try enabling doc\_value and see if this helps (eventually as indexes are created). This is for a central log ELK stack, kibana3 & 4. ES 1.7. Should I add it in more / less places in the template? The template (pastebin: [http://pastebin.com/99h3HPzx](http://pastebin.com/99h3HPzx) ): { "template" : "logstash-", "aliases" : { "alias1" : {}, "dcachebilling" : { "filter" : { "term" : { "type" : "dcache\_billing" } } } }, "settings" : { "index.refresh\_interval…

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [September 30, 2015, 3:23pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423/2 "2015-09-30T15:23:18Z")

</div>

I'd use the first one, if you have specific questions please ask 🙂

---

<div class="post-metadata">

### Author: ![elvarb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elvarb/32/44840_2.png) [@elvarb](https://discuss.elastic.co/u/elvarb)
#### Post date: [September 30, 2015, 3:25pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423/3 "2015-09-30T15:25:03Z")

</div>

Will that then apply to all .raw fields only?

With Elasticsearch 2.0 new default, will that then only apply to .raw fields as well but will integer and timestamp fields also use raw?

(gaddamn I will have to modify lots of dashboards that should use the raw field anyway 🙂 )

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [September 30, 2015, 3:27pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423/4 "2015-09-30T15:27:19Z")

</div>

That's correct, you will need to add doc values to any other fields manually till 2.0 (which isn't far off anyway).

---

<div class="post-metadata">

### Author: ![elvarb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elvarb/32/44840_2.png) [@elvarb](https://discuss.elastic.co/u/elvarb)
#### Post date: [September 30, 2015, 3:38pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423/5 "2015-09-30T15:38:25Z")

</div>

I'm still using Kibana3 and when using the table filter there it doesnt seem to be aware of the .raw field when ordering events in the table. Is there any way to use the .raw field there?

Is it right of me to assume that because of the table filter, just by modifying the @timestamp field for doc\_values would speed things up and save memory?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [September 30, 2015, 3:48pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423/6 "2015-09-30T15:48:53Z")

</div>

> [@elvarb](#):
>
> Is it right of me to assume that because of the table filter, just by modifying the @timestamp field for doc\_values would speed things up and save memory?

It should, yeah.

Might be easier if you raise another thread for the other one to keep things clear 🙂

---

<div class="post-metadata">

### Author: ![elvarb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elvarb/32/44840_2.png) [@elvarb](https://discuss.elastic.co/u/elvarb)
#### Post date: [September 30, 2015, 3:49pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423/7 "2015-09-30T15:49:46Z")

</div>

I will take one step at a time and modify the raw fields first

Thanks for the clarification! 😄

---

<div class="post-metadata">

### Author: ![elvarb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elvarb/32/44840_2.png) [@elvarb](https://discuss.elastic.co/u/elvarb)
#### Post date: [September 30, 2015, 3:54pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423/8 "2015-09-30T15:54:52Z")

</div>

One more question directly related to this.

Should I modify the template already stored in Elasticsearch or should I modify the template specified in Logstash?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [September 30, 2015, 5:21pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423/9 "2015-09-30T17:21:38Z")

</div>

I'd copy the existing LS one and then store it somewhere custom, then manage it in the ES output in the LS conf.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:27am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423/10 "2017-07-06T05:27:37Z")

</div>


