Mapping: trying to ignore 'field' but keeping 'field.raw'

Elastic Stack Elasticsearch
1

We have lots of indexes used for metrics visualizations that grew too much for using the legacy Logstash template (the one with keyword fields named 'raw'). Now I'm creating a new index template to remove the unnecessary features of them, while keeping 100% compatible with old indexes, so our visualizations do not break.

The biggest problem is how to totally disable the analyzed fields while keeping the 'raw' keyword ones. The one option that seems to do the job perfectly is 'enabled:false', but that prevents me from creating the 'raw' subfield to keep the aggregations working.

Based on that I can see about each field in Kibana index pattern, the closest I could get is the following template (snippet):

PUT _template/optimized-logstash
{
  "order": 0,
  "template": "optimized-logstash-*",
  "mappings": {
    "_default_": {
      "dynamic_templates": [
        {
          "string_fields": {
            "mapping": {
              "type": "keyword",
              "index": false,
              "doc_values": false,
              "fields": {
                "raw": {
                  "norms": false,
                  "ignore_above": 256,
                  "type": "keyword"
                }
              }
            },
            "match_mapping_type": "string",
            "match": "*"
          }
        }
      ]
    }
  }
}

POST /optimized-logstash-test/doc
{
  "field": "text value"
}

I disabled all features that I could in order to make it technically "useless", and hopefully without any index overhead. The field is still listed in Kibana's index pattern page, but not searchable nor aggregatable. I can't search for it but I can successfully check for its existence with 'exists:field':

GET /optimized-logstash-test/_search?q=field:value

GET /optimized-logstash-test/_search?q=field.raw:"text value"

GET /optimized-logstash-test/_search?q=_exists_:field

Does that mean I'm still somehow indexing it? How can I make this field totally ignored and still create 'raw' fields?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.