ES 2.0 '_id': 'path' not allowed, but what is allowed?

Elastic Stack Elasticsearch
1

I used GitHub - elastic/elasticsearch-migration: This plugin will help you to check whether you can upgrade directly to the next major version of Elasticsearch, or whether you need to make changes to your data and cluster before doing so. v0.11 on our production index and it flagged our 3 usages of _id with a _path which the docs clearly say will no longer be allowed. However, the wording confuses me as to what is allowed:

The _id field can no longer be configured in new indices. Importantly, it will not be possible to extract the _id value from a path in types, ...

and

(you can't have) "Custom configured _id field."

_id field | Elasticsearch Guide [2.0] | Elastic it says

The _id field is not indexed as its value can be derived automatically from the _uid field.

and

The value of the _id field is accessible in queries and scripts, but not in aggregations or when sorting, where the _uid field should be used instead

My question is in ES 2.0 can we still define a custom type mapping (e.g., 'type': 'string', 'store': True, 'index': 'not_analyzed') and populate our own value of the _id field (just not using path)? On a related note, I notice our current setup using path ends up with some mappings of _id with type string and others with type long, which would appear to be a conflict which is not flagged. Are there global restrictions on the type of the _id field (e.g., can only be 'type': 'string', 'index': 'not_analyzed')?

Please advise.

2

I just stumbled across https://github.com/elastic/elasticsearch/issues/8143 but I'm still not totally clear.

3

You will no longer be able to change _id field type. So do not try to expect long values.

If you want a stored field, use _uid, which is of the format type '#' docid, and extract the doc id.

Please reconsider your use case of path in combination with _id, the chance is high that you can setup your own non-meta-field as a replacement.

4

Ok, so to be clear, the immutable mapping of _id will be:

{
    "type": "string",
    "store": false,
    "index": "not_analyzed"
}

whose value can be populated "as usual" (just not using path). Correct?

5

Quite exact, the locked-down mapper definition is

{
  "type": "string",
  "norms" : { "enabled" : false},
  "analyzer" : "keyword",
  "store" : false
}

see org.elasticsearch.index.mapper.internal.IdFieldMapper

1 Like
6

Thanks very much! Personally, I think this should be in the docs for https://www.elastic.co/guide/en/elasticsearch/reference/2.0/mapping-id-field.html