Custom Template for Logstash 7.10 output

Hello,

i'm new to Elastic Stack and my setup is with Docker and i'm using 7.10.2 version.

For now, i'm trying to analyze my nginx logs in Kibana. I can see my logs in Discover but not in "terms" in the dashboard part. I understood that is because the dynamic mapping of my properties in Elasticsearch is not good (text by default). That is what i'm trying to fix now with a custom mapping.

My logstash output is like this:

output {
	elasticsearch {
		hosts => "elasticsearch:9200"
		user => "elastic"
		password => "changeme"
		ecs_compatibility => disabled
		index => "logstash-%{+YYYY.MM.dd}"
		manage_template => true
		template_overwrite => true
		template => "/etc/logstash/conf.d/es_template.json"
	}
}

My es_template.json looks like that (i just put few properties to test):

  {
  "index_patterns": [
    "logstash*"
  ],
  "template": {
    "mappings": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "method": {
            "type": "keyword"
          },
          "body_bytes_sent": {
            "type": "integer"
          },
          "visitor_ip": {
            "type": "ip"
          }
        }
    }
  }
}

but when i check the mapping at localhost:9200/logstash-*/_mapping?pretty

i have the default mapping and my custom mapping is not taken into account. For example for the ip, it's still text and not IP type:

"visitor_ip" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    }

When checking the logs, it seems that the template is integrated:

logstash container:
[logstash.outputs.elasticsearch][main] Using mapping template from {:path=>"/etc/logstash/conf.d/es_template.json"}
Installing elasticsearch template to _template/logstash
elasticsearch:
"message": "adding template [logstash] for index patterns [logstash*]",

I think the problem is with the format of the JSON mapping file but i cannot figure what is wrong. I tried different ways none is working.

For information, If i add:

`"priority": 200,`

to my template i have a loading error of the template in the logstash container logs.

I also tried to read the official docs many times, but it's not very clear to me.

Thank you very much for any help

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.