ES Mapping template taking "index : analyzed" despite I am using "index : not_analyzed"

Elastic Stack Elasticsearch
1

Hello Experts,

I am using below ES/logstash template to create a mapping. I used "index : not_analyzed" in my template but after index creation when I look it via kibana/curl I am getting index as analyzed. Why did index set as analyzed? Can someone help me to fix this issue?

ES Template

{
"template" : "apacheaccesslog",
"settings" : { "index.refresh_interval" : "60s" },
"mappings" : {
"default" : {
"_all" : { "enabled" : false },
"dynamic_templates" : [{
"message_field" : {
"match" : "message",
"match_mapping_type" : "string",
"mapping" : { "type" : "string", "index" : "not_analyzed" }
}
}, {
"string_fields" : {
"match" : "*",
"match_mapping_type" : "string",
"mapping" : { "type" : "string", "index" : "not_analyzed" }
}
}],
"properties" : {
"@timestamp" : { "type" : "date", "format" : "dateOptionalTime" },
"@version" : { "type" : "integer", "index" : "not_analyzed" },
"agent" : { "type" : "string", "index" : "not_analyzed" },
"bytes" : { "type" : "long", "norms" : { "enabled" : false } },
"host" : { "type" : "string", "index" : "not_analyzed" },
"clientip" : { "type" : "ip", "norms" : { "enabled" : false } },
"httpversion" : { "type" : "float" },
"referrer" : { "type" : "string", "index" : "not_analyzed" },
"request" : { "type" : "string", "index" : "not_analyzed", "include_in_all": false },
"response" : { "type" : "integer", "index" : "not_analyzed" },
"geoip" : { "type" : "object", "dynamic" : true, "path" : "full", "properties" : { "location" : { "type" : "geo_point" } } },
"verb" : { "type" : "string", "norms" : { "enabled" : false } }
}
}
}
}

Logstash output part

output {
elasticsearch {
host => "192.168.1.24"
cluster => "remcal"
protocol => "http"
index => "apacheaccesslog-%{+YYYY.MM.dd}"
template => "/etc/elasticsearch/templates/apacheaccess.json"
template_name => "apacheaccesslog"
template_overwrite => true
}
}

ES Mapping after index creation

[root@peter templates]# curl -XGET 'http://localhost:9200/apacheaccesslog-2015.08.31/_mapping?pretty=true
'
{
"apache-accesslog-2015.08.31" : {
"mappings" : {
"apache_access" : {
"properties" : {
"@timestamp" : {
"type" : "date",
"format" : "dateOptionalTime"
},
"@version" : {
"type" : "string"
},
"agent" : {
"type" : "string"
},
"agent.device" : {
"type" : "string"
},
"agent.name" : {
"type" : "string"
},
"agent.os" : {
"type" : "string"
},
"agent.os_name" : {
"type" : "string"
},
"auth" : {
"type" : "string"
},
"bytes" : {
"type" : "string"
},
"clientip" : {
"type" : "string"
},
"host" : {
"type" : "string"
},
"httpversion" : {
"type" : "string"
},
"ident" : {
"type" : "string"
},
"message" : {
"type" : "string"
},
"path" : {
"type" : "string"
},
"referrer" : {
"type" : "string"
},
.......
},
"verb" : {
"type" : "string"
}
}
}
}
}
}

ES Version

[root@peter templates]# rpm -q elasticsearch
elasticsearch-1.7.1-1.noarch

Regards,
Peter

3

Hello,

Can anyone help me with this?

Regards,
Peter

4

The template you have provided does not contain a wildcard to make it match the index pattern specified in the logstash config. Try changing this to "template" : "apacheaccesslog"* instead and it should apply correctly for newly created indices.

5

Thanks Christian,

Let me check and get back to you

Regards,
Peter

6

@Christian_Dahlqvist,

Thanks, finally the issue got solved by following your suggestion. I knew I made a minor mistake but could't figure it out.

Regards,
Peter