# Missing fields in .security templates

**URL:** <https://discuss.elastic.co/t/missing-fields-in-security-templates/220678>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-security\
**Created:** [February 24, 2020, 2:42pm UTC](https://discuss.elastic.co/t/missing-fields-in-security-templates/220678 "2020-02-24T14:42:01Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![kannan\_raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kannan_raj/32/63201_2.png) [@kannan\_raj](https://discuss.elastic.co/u/kannan_raj)\
**Post date:** [February 24, 2020, 2:42pm UTC](https://discuss.elastic.co/t/missing-fields-in-security-templates/220678/1 "2020-02-24T14:42:01Z")

</div>

Hi,

I have tried to enable native realm authentication in elasticsearch 7.3.0 but the problem i have noticed that I could see missing values in .security index mapping when ES cluster is running without kibana. As a result i could not able to create any user / roles by using API.

log

> [2020-02-24T11:43:33,609][INFO][o.e.x.s.s.SecurityIndexManager] Missing \_meta field in mapping [\_doc] of index [.security]  
> [2020-02-24T11:43:33,609][WARN][o.e.c.s.ClusterApplierService] failed to notify ClusterStateListener  
> java.lang.IllegalStateException: Cannot read security-version string in index .security  
> at org.elasticsearch.xpack.security.support.SecurityIndexManager.readMappingVersion(SecurityIndexManager.java:320) ~[?:?]

I have tested the same ES configuration with kibana pointed to ES. I didn't face issues.

.security index mapping without kibana

> {  
> ".security-7" : {  
> "aliases" : {  
> ".security" : { }  
> },  
> "mappings" : {  
> "dynamic\_templates" : [  
> {  
> "message\_field" : {  
> "path\_match" : "message",  
> "match\_mapping\_type" : "string",  
> "mapping" : {  
> "norms" : false,  
> "type" : "text"  
> }  
> }  
> },  
> {  
> "string\_fields" : {  
> "match" : "\*",  
> "match\_mapping\_type" : "string",  
> "mapping" : {  
> "fields" : {  
> "keyword" : {  
> "ignore\_above" : 256,  
> "type" : "keyword"  
> }  
> },  
> "norms" : false,  
> "type" : "text"  
> }  
> }  
> }  
> ],  
> "properties" : {  
> "@timestamp" : {  
> "type" : "date"  
> },  
> "@version" : {  
> "type" : "keyword"  
> },  
> "actions" : {  
> "type" : "text",  
> "norms" : false,  
> "fields" : {  
> "keyword" : {  
> "type" : "keyword",  
> "ignore\_above" : 256  
> }  
> }  
> },  
> "application" : {  
> "type" : "text",  
> "norms" : false,  
> "fields" : {  
> "keyword" : {  
> "type" : "keyword",  
> "ignore\_above" : 256  
> }  
> }  
> },  
> "geoip" : {  
> "dynamic" : "true",  
> "properties" : {  
> "ip" : {  
> "type" : "ip"  
> },  
> "latitude" : {  
> "type" : "half\_float"  
> },  
> "location" : {  
> "type" : "geo\_point"  
> },  
> "longitude" : {  
> "type" : "half\_float"  
> }  
> }  
> },  
> "indices" : {  
> "properties" : {  
> "allow\_restricted\_indices" : {  
> "type" : "boolean"  
> },  
> "names" : {  
> "type" : "text",  
> "norms" : false,  
> "fields" : {  
> "keyword" : {  
> "type" : "keyword",  
> "ignore\_above" : 256  
> }  
> }  
> },  
> "privileges" : {  
> "type" : "text",  
> "norms" : false,  
> "fields" : {  
> "keyword" : {  
> "type" : "keyword",  
> "ignore\_above" : 256  
> }  
> }  
> }  
> }  
> },  
> "metadata" : {  
> "type" : "object"  
> },  
> "name" : {  
> "type" : "text",  
> "norms" : false,  
> "fields" : {  
> "keyword" : {  
> "type" : "keyword",  
> "ignore\_above" : 256  
> }  
> }  
> },  
> "type" : {  
> "type" : "text",  
> "norms" : false,  
> "fields" : {  
> "keyword" : {  
> "type" : "keyword",  
> "ignore\_above" : 256  
> }  
> }  
> }  
> }  
> },  
> "settings" : {  
> "index" : {  
> "refresh\_interval" : "5s",  
> "number\_of\_shards" : "1",  
> "auto\_expand\_replicas" : "0-1",  
> "provided\_name" : ".security-7",  
> "format" : "6",  
> "creation\_date" : "1582532501252",  
> "analysis" : {  
> "filter" : {  
> "email" : {  
> "type" : "pattern\_capture",  
> "preserve\_original" : "true",  
> "patterns" : [  
> "([^@]+)",  
> "(\p{L}+)",  
> "(\d+)",  
> "@(.+)"  
> ]  
> }  
> },  
> "analyzer" : {  
> "email" : {  
> "filter" : [  
> "email",  
> "lowercase",  
> "unique"  
> ],  
> "tokenizer" : "uax\_url\_email"  
> }  
> }  
> },  
> "priority" : "1000",  
> "number\_of\_replicas" : "1",  
> "uuid" : "Nv2KiMKDRfW8UgWL1hANbw",  
> "version" : {  
> "created" : "7030099"  
> }  
> }  
> }  
> }  
> }

.security index mapping with kibana

> {  
> ".security-7" : {  
> "aliases" : {  
> ".security" : { }  
> },  
> "mappings" : {  
> "dynamic" : "strict",  
> "\_meta" : {  
> "security-version" : "7.3.0"  
> },  
> "properties" : {  
> "access\_token" : {  
> "properties" : {  
> "invalidated" : {  
> "type" : "boolean"  
> },  
> "realm" : {  
> "type" : "keyword"  
> },  
> "user\_token" : {  
> "properties" : {  
> "authentication" : {  
> "type" : "binary"  
> },  
> "expiration\_time" : {  
> "type" : "date",  
> "format" : "epoch\_millis"  
> },  
> "id" : {  
> "type" : "keyword"  
> },  
> "metadata" : {  
> "type" : "object",  
> "dynamic" : "false"  
> },  
> "version" : {  
> "type" : "integer"  
> }  
> }  
> }  
> }  
> },  
> "actions" : {  
> "type" : "keyword"  
> },  
> "api\_key\_hash" : {  
> "type" : "keyword",  
> "index" : false,  
> "doc\_values" : false  
> },  
> "api\_key\_invalidated" : {  
> "type" : "boolean"  
> },  
> "application" : {  
> "type" : "keyword"  
> },  
> "applications" : {  
> "properties" : {  
> "application" : {  
> "type" : "keyword"  
> },  
> "privileges" : {  
> "type" : "keyword"  
> },  
> "resources" : {  
> "type" : "keyword"  
> }  
> }  
> },  
> "cluster" : {  
> "type" : "keyword"  
> },  
> "creation\_time" : {  
> "type" : "date",  
> "format" : "epoch\_millis"  
> },  
> "creator" : {  
> "properties" : {  
> "metadata" : {  
> "type" : "object",  
> "dynamic" : "false"  
> },  
> "principal" : {  
> "type" : "keyword"  
> },  
> "realm" : {  
> "type" : "keyword"  
> }  
> }  
> },  
> "doc\_type" : {  
> "type" : "keyword"  
> },  
> "email" : {  
> "type" : "text",  
> "analyzer" : "email"  
> },  
> "enabled" : {  
> "type" : "boolean"  
> },  
> "expiration\_time" : {  
> "type" : "date",  
> "format" : "epoch\_millis"  
> },  
> "full\_name" : {  
> "type" : "text"  
> },  
> "global" : {  
> "properties" : {  
> "application" : {  
> "properties" : {  
> "manage" : {  
> "properties" : {  
> "applications" : {  
> "type" : "keyword"  
> }  
> }  
> }  
> }  
> }  
> }  
> },  
> "indices" : {  
> "properties" : {  
> "allow\_restricted\_indices" : {  
> "type" : "boolean"  
> },  
> "field\_security" : {  
> "properties" : {  
> "except" : {  
> "type" : "keyword"  
> },  
> "grant" : {  
> "type" : "keyword"  
> }  
> }  
> },  
> "names" : {  
> "type" : "keyword"  
> },  
> "privileges" : {  
> "type" : "keyword"  
> },  
> "query" : {  
> "type" : "keyword"  
> }  
> }  
> },  
> "limited\_by\_role\_descriptors" : {  
> "type" : "object",  
> "enabled" : false  
> },  
> "metadata" : {  
> "type" : "object",  
> "dynamic" : "false"  
> },  
> "name" : {  
> "type" : "keyword"  
> },  
> "password" : {  
> "type" : "keyword",  
> "index" : false,  
> "doc\_values" : false  
> },  
> "refresh\_token" : {  
> "properties" : {  
> "client" : {  
> "properties" : {  
> "realm" : {  
> "type" : "keyword"  
> },  
> "type" : {  
> "type" : "keyword"  
> },  
> "user" : {  
> "type" : "keyword"  
> }  
> }  
> },  
> "invalidated" : {  
> "type" : "boolean"  
> },  
> "refresh\_time" : {  
> "type" : "date",  
> "format" : "epoch\_millis"  
> },  
> "refreshed" : {  
> "type" : "boolean"  
> },  
> "superseding" : {  
> "properties" : {  
> "encrypted\_tokens" : {  
> "type" : "binary"  
> },  
> "encryption\_iv" : {  
> "type" : "binary"  
> },  
> "encryption\_salt" : {  
> "type" : "binary"  
> }  
> }  
> },  
> "token" : {  
> "type" : "keyword"  
> }  
> }  
> },  
> "role\_descriptors" : {  
> "type" : "object",  
> "enabled" : false  
> },  
> "role\_templates" : {  
> "properties" : {  
> "format" : {  
> "type" : "keyword"  
> },  
> "template" : {  
> "type" : "text"  
> }  
> }  
> },  
> "roles" : {  
> "type" : "keyword"  
> },  
> "rules" : {  
> "type" : "object",  
> "dynamic" : "false"  
> },  
> "run\_as" : {  
> "type" : "keyword"  
> },  
> "type" : {  
> "type" : "keyword"  
> },  
> "username" : {  
> "type" : "keyword"  
> },  
> "version" : {  
> "type" : "integer"  
> }  
> }  
> },  
> "settings" : {  
> "index" : {  
> "number\_of\_shards" : "1",  
> "auto\_expand\_replicas" : "0-1",  
> "provided\_name" : ".security-7",  
> "format" : "6",  
> "creation\_date" : "1566300744986",  
> "analysis" : {  
> "filter" : {  
> "email" : {  
> "type" : "pattern\_capture",  
> "preserve\_original" : "true",  
> "patterns" : [  
> "([^@]+)",  
> "(\p{L}+)",  
> "(\d+)",  
> "@(.+)"  
> ]  
> }  
> },  
> "analyzer" : {  
> "email" : {  
> "filter" : [  
> "email",  
> "lowercase",  
> "unique"  
> ],  
> "tokenizer" : "uax\_url\_email"  
> }  
> }  
> },  
> "priority" : "1000",  
> "number\_of\_replicas" : "1",  
> "uuid" : "AaA\_zblDSxSkm2mDLpxpkg",  
> "version" : {  
> "created" : "7030099"  
> }  
> }  
> }  
> }  
> }

elasticsearch.yml

> cluster.name: cluster  
> node.name: node1  
> path.data:  
> - /data  
> path.logs: /var/log/elasticsearch  
> bootstrap.memory\_lock: true  
> network.host: ip  
> discovery.seed\_providers: file  
> cluster.initial\_master\_nodes: ["node1", "node2", "node3"]  
> action.destructive\_requires\_name: true  
> xpack.monitoring.enabled: false  
> xpack.monitoring.collection.enabled: false  
> xpack.security.enabled: true  
> xpack.security.authc.realms.file.csc\_file.order: 0  
> xpack.security.http.ssl.enabled: true  
> xpack.security.http.ssl.key: "key.key"  
> xpack.security.http.ssl.certificate: "cert.crt"  
> xpack.security.http.ssl.verification\_mode: "certificate"  
> xpack.security.http.ssl.certificate\_authorities: ["ca.crt"]  
> xpack.security.http.ssl.supported\_protocols: TLSv1.2  
> xpack.security.transport.ssl.enabled: true  
> xpack.security.transport.ssl.key: "key.key"  
> xpack.security.transport.ssl.certificate: "cert.crt"  
> xpack.security.transport.ssl.verification\_mode: "certificate"  
> xpack.security.transport.ssl.certificate\_authorities: ["ca.crt"]  
> xpack.security.transport.ssl.supported\_protocols: TLSv1.2

---

<div class="post-metadata">

**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\
**Post date:** [February 24, 2020, 2:52pm UTC](https://discuss.elastic.co/t/missing-fields-in-security-templates/220678/2 "2020-02-24T14:52:55Z")

</div>

The most probable cause is that you have an index template that applies to `*` and as such is applied to `.security` also when it is auto-created. The problem is described in [this issue](https://github.com/elastic/elasticsearch/pull/44918) and it was [resolved in 7.4.0](https://github.com/elastic/elasticsearch/pull/45120) . If you can't upgrade to \> 7.4.0 for some reason, you need to ensure that you fix your templates to have smaller scope and not match `.security`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 23, 2020, 2:52pm UTC](https://discuss.elastic.co/t/missing-fields-in-security-templates/220678/3 "2020-03-23T14:52:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.