Custom Template for Logstash 7.10 output

Elastic Stack Logstash
1

Hello,

i'm new to Elastic Stack and my setup is with Docker and i'm using 7.10.2 version.

For now, i'm trying to analyze my nginx logs in Kibana. I can see my logs in Discover but not in "terms" in the dashboard part. I understood that is because the dynamic mapping of my properties in Elasticsearch is not good (text by default). That is what i'm trying to fix now with a custom mapping.

My logstash output is like this:

output {
	elasticsearch {
		hosts => "elasticsearch:9200"
		user => "elastic"
		password => "changeme"
		ecs_compatibility => disabled
		index => "logstash-%{+YYYY.MM.dd}"
		manage_template => true
		template_overwrite => true
		template => "/etc/logstash/conf.d/es_template.json"
	}
}

My es_template.json looks like that (i just put few properties to test):

  {
  "index_patterns": [
    "logstash*"
  ],
  "template": {
    "mappings": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "method": {
            "type": "keyword"
          },
          "body_bytes_sent": {
            "type": "integer"
          },
          "visitor_ip": {
            "type": "ip"
          }
        }
    }
  }
}

but when i check the mapping at localhost:9200/logstash-*/_mapping?pretty

i have the default mapping and my custom mapping is not taken into account. For example for the ip, it's still text and not IP type:

"visitor_ip" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    }

When checking the logs, it seems that the template is integrated:

logstash container:
[logstash.outputs.elasticsearch][main] Using mapping template from {:path=>"/etc/logstash/conf.d/es_template.json"}
Installing elasticsearch template to _template/logstash
elasticsearch:
"message": "adding template [logstash] for index patterns [logstash*]",

I think the problem is with the format of the JSON mapping file but i cannot figure what is wrong. I tried different ways none is working.

For information, If i add:

`"priority": 200,`

to my template i have a loading error of the template in the logstash container logs.

I also tried to read the official docs many times, but it's not very clear to me.

Thank you very much for any help

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.