I have tried to create custom index pattern in Elasticsearch based on my input data. So I wrote some custom json and point it to template section of logstash output plugin.
But not sure why the index are not created in ELS as per the json file. It just creating default pattern in ELS.
Output section:
}
Do I need to add this json file somewhere else.?.
Version Details:
msimos
July 17, 2015, 12:17am
2
Did you delete the old template from Elasticsearch? Logstash won't add the new one until you delete the old one from Elasticsearch. Use:
curl -XDELETE <http://localhost:9200/_template/OldTemplateName?pretty>
Or you can use template_overwrite:
https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-template_overwrite
warkolm
July 17, 2015, 2:34am
3
Also, is the template valid json?
Why do we need to delete the old template[ default template] from Elasticsearch ? Do you mean, we couldn't create custom template per index ?.
@warkolm , Good to see you back again to respond for my query.
Yes, its a valid json file. I have cross verified my json file on this link http://jsonlint.com/ .
warkolm
July 17, 2015, 4:58am
6
Right, but is it valid Elasticsearch json
Yes , I already start working with the mentioned json file using curl PUT command. its working fine. Not sure , why this json file is not accepting in logstash output plugin.
msimos
July 17, 2015, 5:53pm
8
If you do:
curl -XGET http://localhost:9200/_template/TemplateName?pretty
What do you get? Replace TemplateName with what you specified as the template name in elasticsearch-new.json.
I'm having the same problem. This is my template printed:
{
"sigas-log-logstash": {
"order": 0,
"template": "sigas-log-logstash",
"settings": {
"index": {
"refresh_interval": "60s"
}
},
"mappings": {
"_default_": {
"dynamic_templates": [
{
"message_field": {
"mapping": {
"index": "not_analyzed",
"type": "string"
},
"match_mapping_type": "string",
"match": "message"
}
},
{
"string_fields": {
"mapping": {
"index": "not_analyzed",
"type": "string"
},
"match_mapping_type": "string",
"match": "*"
}
}
],
"_all": {
"enabled": false
},
"properties": {
"severity": {
"index": "not_analyzed",
"type": "string"
},
"date": {
"format": "yyyyMMdd HHmmss.SSSSSS",
"type": "date"
},
"interpreter": {
"index": "not_analyzed",
"type": "string"
},
"thread": {
"index": "not_analyzed",
"type": "integer"
},
"uuid": {
"index": "not_analyzed",
"type": "string"
},
"received_from": {
"index": "not_analyzed",
"type": "string"
},
"@timestamp": {
"format": "dateOptionalTime",
"type": "date"
},
"resource_id_module": {
"index": "not_analyzed",
"type": "string"
},
"file": {
"index": "not_analyzed",
"type": "string"
},
"execute_time": {
"index": "not_analyzed",
"type": "integer"
},
"@version": {
"index": "not_analyzed",
"type": "integer"
},
"log_message": {
"index": "not_analyzed",
"type": "string"
},
"id_log": {
"index": "not_analyzed",
"type": "string"
}
}
}
},
"aliases": {}
}
}
