# Logstash default template for elasticsearch output **URL:** \ **Category:** Logstash\ **Created:** [July 29, 2015, 8:45pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526 "2015-07-29T20:45:04Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![pavan\_bkv](https://avatars.discourse-cdn.com/v4/letter/p/6a8cbe/32.png) [@pavan\_bkv](https://discuss.elastic.co/u/pavan_bkv)\ **Post date:** [July 29, 2015, 8:45pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/1 "2015-07-29T20:45:04Z")
So using logstash exec input (JSON response) to output to an elasticsearch cluster (using the default mapping template). Somehow even though the resultant EXEC event has a @timestamp, my mapping misses it and hence making it useless for my kibana! Can someone help me understand what I am missing? Basically my EXEC input just makes a rest call to get another ES cluster stats and tries to just store the entire JSON into another ES cluster which has kibana built on top of it! The problem is that Kibana won't recognize the index since its missing the @timestamp in the mapping (but the data does exist) ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 29, 2015, 9:38pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/2 "2015-07-29T21:38:56Z")
You probably want a date filter to force LS to use that field. ---
**Author:** ![pavan\_bkv](https://avatars.discourse-cdn.com/v4/letter/p/6a8cbe/32.png) [@pavan\_bkv](https://discuss.elastic.co/u/pavan_bkv)\ **Post date:** [July 29, 2015, 10:06pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/3 "2015-07-29T22:06:41Z")
Mark as I mentioned its already there in the source. Here is a snippet shwing it from the ES indices stats I am trying to capture (JSON response) "\_source": { "\_shards": { "total": 42, "successful": 21, "failed": 0 }, .... .... "@version": "1", **"@timestamp": "2015-07-29T19:35:49.557Z",** "type": "esl\_addr\_cache\_es\_qa\_index\_stats", "event\_time": "2015-07-29T19:35:49.557Z", "host": "0.0.0.0", "command": "curl -s -X GET [http://someip:9200/\_stats](http://someip:9200/_stats)" } ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 29, 2015, 10:10pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/4 "2015-07-29T22:10:40Z")
Right but it won't just take that and use it just because it is there. You need to use a date filter. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 29, 2015, 10:11pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/5 "2015-07-29T22:11:28Z")
Also, your cluster is open to the internet. THIS IS BAD. ---
**Author:** ![daks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daks/32/3883_2.png) [@daks](https://discuss.elastic.co/u/daks)\ **Post date:** [July 30, 2015, 2:25pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/6 "2015-07-30T14:25:22Z")
In fact. @pavan_bkv: all your data is available online by anybody. ---
**Author:** ![pavan\_bkv](https://avatars.discourse-cdn.com/v4/letter/p/6a8cbe/32.png) [@pavan\_bkv](https://discuss.elastic.co/u/pavan_bkv)\ **Post date:** [July 30, 2015, 9:28pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/7 "2015-07-30T21:28:40Z")
Mark that is fine, its just a play area for me. Anyhow isn't it a basic function for a filter in LS to actually filter an EVENT or do some data transformation? I still don't understand how adding a filter will help. I did try doing this but still it did not work! filter { if [type] == "esl\_addr\_cache\_es\_qa\_index\_stats" { date { match =\> ["%{@timestamp}", "YYYY-MM-dd HH:mm:ss"] add\_field =\> ["event\_time", "%{@timestamp}"] } } } ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 30, 2015, 9:57pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/8 "2015-07-30T21:57:48Z")
Yes LS does do filtering, but it's up to you to tell it what to do otherwise it will just pass through what it gets without any intelligence. If you post your entire config it might help. ---
**Author:** ![pavan\_bkv](https://avatars.discourse-cdn.com/v4/letter/p/6a8cbe/32.png) [@pavan\_bkv](https://discuss.elastic.co/u/pavan_bkv)\ **Post date:** [July 30, 2015, 10:00pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/9 "2015-07-30T22:00:59Z")
Here is my config (pay attention to exec input, its a json response of another ES index stats). LS default adds the @timestamp. I can even see it in my source document, just that the mapping is missing it! **here is my config:** # Input using lumberjack (a.k.a logstash forwarder) # Output using elasticsearch input { ``` # Elasticsearch indices stats for ESL Address Cache - QA exec { command => "curl -s -X GET http://someesnode:9200/_stats" codec => "json" interval => 60 type => "esl_addr_cache_es_qa_index_stats" } ``` } filter { if [type] == "esl\_addr\_cache\_es\_qa\_index\_stats" { date { match =\> ["%{@timestamp}", "YYYY-MM-dd HH:mm:ss"] add\_field =\> ["event\_time", "%{@timestamp}"] } } } output { if [type] == "esl\_addr\_cache\_es\_qa\_index\_stats" { elasticsearch { bind\_host =\> "someip" index =\> "es-stats-logstash-%{+YYYY.MM.dd}" cluster =\> "log\_aggr\_es\_cluster" codec =\> "json" } } } ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 30, 2015, 10:59pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/10 "2015-07-30T22:59:23Z")
Ok now I understand! 😄 It should be using that @timestamp given it's LS that is actually generating it. What does the mapping for the index look like? ---
**Author:** ![pavan\_bkv](https://avatars.discourse-cdn.com/v4/letter/p/6a8cbe/32.png) [@pavan\_bkv](https://discuss.elastic.co/u/pavan_bkv)\ **Post date:** [August 3, 2015, 5:10pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/11 "2015-08-03T17:10:25Z")
Mark I can't put the entire mapping here, is there a way to attach as a file? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 3, 2015, 9:05pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/12 "2015-08-03T21:05:06Z")
Put it in gist/pastebin/etc and link to it. ---
**Author:** ![pavan\_bkv](https://avatars.discourse-cdn.com/v4/letter/p/6a8cbe/32.png) [@pavan\_bkv](https://discuss.elastic.co/u/pavan_bkv)\ **Post date:** [August 3, 2015, 9:38pm UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/13 "2015-08-03T21:38:59Z")
Thanks Mark. Fixed the issue with custom mapping { "template": "es-stats-logstash-\*", "settings": { "index.refresh\_interval": "5s" }, "mappings": { "esl\_addr\_cache\_es\_qa\_index\_stats": { "properties": { "@timestamp": { "format": "dateOptionalTime", "type": "date", "index" : "analyzed" } } } } } ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:33am UTC](https://discuss.elastic.co/t/logstash-default-template-for-elasticsearch-output/26526/14 "2017-07-06T05:33:03Z")