Disable field based on its name

Elastic Stack Elasticsearch
1

Hi,

I'm using ElasticSearch 1.5 and using the bulk API to index a bunch of data.

I want to get rid of some fields that are present in the documents that are being indexed.

I'm using a template to create the mappings and right now I'm adding those fields as disabled fields like this:

"properties: {
  "ns_used_0": { "enabled": false },
  "ns_used_1": { "enabled": false },
  "ns_used_2": { "enabled": false },
  "ns_used_3": { "enabled": false },
  "ns_used_4": { "enabled": false },
  "ns_used_5": { "enabled": false },
  [ more... ]
}

But I don't really want to do that because that list can grow a lot in the future.
I want to disable every field that starts with "ns_".

I thought of using dynamic templates like this:

"dynamic_templates": [
  {
    "ignored": {
      "match": "ns_*",
      "match_mapping_type": "string",
      "mapping":   { "enabled": "false" }
    }
  }
]

But that is not working for me. I can see in Kibana that the fields are being added.

Am I missing something? Is there any other way to do that?

2

Try changing "mapping": {"enabled": "false"} to "mapping": { "index: "no" }

Does that help?

3

That works with dynamic_templates, but unfortunately, it doesn't have exactly the same effect.

I suppose that if we set index to no, the field won't be present in the index and I won't be able to search documents based on that field, but it's present on ES, Kibana knows about it, and I can get their value once I've found a document.

However, when the document is disabled, I can't search it and I can't see it in Kibana nor in ES documents.

Not sure what's the exactly internal difference in terms of performance, though.

Thanks!

4

You could set "stored" to "no" as well and make sure to exclude the fields
from _source, see:
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html#include-exclude