Elasticsearch enable doc_values for all non-analyzed fields in template and re-index


(deepak deore) #1

ES version: 1.5.2 (amazon elasticearch service)

How do I enable doc values for all non-analyzed fields in template from API ? So that indices being crated will have doc values enabled. After this how do I re-index old indices from API which will enable doc values for old indices also.

Here is my template:

$ curl -s 'http://AWS_ES/_template?pretty=true/'
{
  "logstash" : {
    "order" : 0,
    "template" : "logstash-*",
    "settings" : {
      "index.refresh_interval" : "5s"
    },
    "mappings" : {
      "_default_" : {
        "dynamic_templates" : [ {
          "message_field" : {
            "mapping" : {
              "index" : "analyzed",
              "omit_norms" : true,
              "type" : "string"
            },
            "match_mapping_type" : "string",
            "match" : "message"
          }
        }, {
          "string_fields" : {
            "mapping" : {
              "index" : "analyzed",
              "omit_norms" : true,
              "type" : "string",
              "fields" : {
                "raw" : {
                  "ignore_above" : 256,
                  "index" : "not_analyzed",
                  "type" : "string"
                }
              }
            },
            "match_mapping_type" : "string",
            "match" : "*"
          }
        } ],
        "_all" : {
          "omit_norms" : true,
          "enabled" : true
        },
        "properties" : {
          "geoip" : {
            "dynamic" : true,
            "type" : "object",
            "properties" : {
              "location" : {
                "type" : "geo_point"
              }
            }
          },
          "@version" : {
            "index" : "not_analyzed",
            "type" : "string"
          }
        }
      }
    },
    "aliases" : { }
  }
}

(Magnus B├Ąck) #2

You already know how to extract the existing index template, so modify the template and update it with a PUT operation. Keep in mind that Logstash with the default settings will manage the logstash-* index template for you so you might want to disable that if you want to manage them yourself, or you can reconfigure Logstash to use a different index template file than the default. See the options of the elasticsearch output.

To reindex the documents to a new index you can use Logstash itself, es-reindex, or a custom script written in e.g. Python. Keep in mind that indexes can't be renamed, so you have to reindex into an index with new name, e.g. based on the original name but with a suffix. After deleting the original index you can create an index alias that allows you to use the index under the old name.


(system) #3