Logstash not using given template json file on output section

Elastic Stack Logstash
1

I have tried to create custom index pattern in Elasticsearch based on my input data. So I wrote some custom json and point it to template section of logstash output plugin.

But not sure why the index are not created in ELS as per the json file. It just creating default pattern in ELS.

Output section:
output {
stdout {codec => rubydebug }
elasticsearch {
host => localhost
protocol => http
index => "new-%{+YYYY.MM.dd}"
template => "/opt/logstash/elasticsearch-new.json"

}

Do I need to add this json file somewhere else.?.

Version Details:
Logstash : 1.5.2
ELS: .13.9

2

Did you delete the old template from Elasticsearch? Logstash won't add the new one until you delete the old one from Elasticsearch. Use:

curl -XDELETE <http://localhost:9200/_template/OldTemplateName?pretty>

Or you can use template_overwrite:

https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-template_overwrite

3

Also, is the template valid json?

4

Why do we need to delete the old template[ default template] from Elasticsearch ? Do you mean, we couldn't create custom template per index ?.

5

@warkolm , Good to see you back again to respond for my query.

Yes, its a valid json file. I have cross verified my json file on this link http://jsonlint.com/ .

6

Right, but is it valid Elasticsearch json :smile:
Can you apply the template using Sense or curl or similar?

7

Yes , I already start working with the mentioned json file using curl PUT command. its working fine. Not sure , why this json file is not accepting in logstash output plugin.

8

If you do:

curl -XGET http://localhost:9200/_template/TemplateName?pretty

What do you get? Replace TemplateName with what you specified as the template name in elasticsearch-new.json.

9

I'm having the same problem. This is my template printed:

{
  "sigas-log-logstash": {
    "order": 0,
    "template": "sigas-log-logstash",
    "settings": {
      "index": {
        "refresh_interval": "60s"
      }
    },
    "mappings": {
      "_default_": {
        "dynamic_templates": [
          {
            "message_field": {
              "mapping": {
                "index": "not_analyzed",
                "type": "string"
              },
              "match_mapping_type": "string",
              "match": "message"
            }
          },
          {
            "string_fields": {
              "mapping": {
                "index": "not_analyzed",
                "type": "string"
              },
              "match_mapping_type": "string",
              "match": "*"
            }
          }
        ],
        "_all": {
          "enabled": false
        },
        "properties": {
          "severity": {
            "index": "not_analyzed",
            "type": "string"
          },
          "date": {
            "format": "yyyyMMdd HHmmss.SSSSSS",
            "type": "date"
          },
          "interpreter": {
            "index": "not_analyzed",
            "type": "string"
          },
          "thread": {
            "index": "not_analyzed",
            "type": "integer"
          },
          "uuid": {
            "index": "not_analyzed",
            "type": "string"
          },
          "received_from": {
            "index": "not_analyzed",
            "type": "string"
          },
          "@timestamp": {
            "format": "dateOptionalTime",
            "type": "date"
          },
          "resource_id_module": {
            "index": "not_analyzed",
            "type": "string"
          },
          "file": {
            "index": "not_analyzed",
            "type": "string"
          },
          "execute_time": {
            "index": "not_analyzed",
            "type": "integer"
          },
          "@version": {
            "index": "not_analyzed",
            "type": "integer"
          },
          "log_message": {
            "index": "not_analyzed",
            "type": "string"
          },
          "id_log": {
            "index": "not_analyzed",
            "type": "string"
          }
        }
      }
    },
    "aliases": {}
  }
}