Can not reduce index size in elasticsearch

Elastic Stack Elasticsearch
1

Version: Elasticsearch 5.4

Hi, I am a new comer to elasticsearch , and use it to analyze my nginx access log, before I use the default mapping template, and now I want to reduce index size followed by some rules below:

  • disable _all field
  • remove text mapping, and only map keyword type for all the string fileds

Then I update the index template, use reindex API to create a new index, the result is that the mapped fields are decreased, but the index size is increased. I can not tell this reason. How to reduce the size of my index correctly? Thank you for your time.

Here is my template

{
  "template": "*nginx*",
  "aliases": {},
  "mappings": {
    "_default_": {
      "properties": {
        "@version": {
          "type": "keyword",
          "include_in_all": false
        },
        "geoip": {
          "properties": {
            "longitude": {
              "type": "half_float"
            },
            "location": {
              "type": "geo_point"
            },
            "latitude": {
              "type": "half_float"
            },
            "ip": {
              "type": "ip"
            }
          },
          "dynamic": true
        },
        "@timestamp": {
          "type": "date",
          "include_in_all": false
        }
      },
      "_all": {
        "enabled": false
      },
      "dynamic_templates": [
        {
          "string_fields": {
            "match_mapping_type": "string",
            "mapping": {
              "type": "keyword",
              "norms": false
            }
          }
        }
      ]
    }
  },
  "settings": {
    "index": {
      "refresh_interval": "30s"
    }
  },
  "version": 1,
  "order": 0
}

And the diff to the default one is

...
"dynamic_templates": [
          {
            "message_field": {
              "mapping": {
                "type": "text",
                "norms": false
              },
              "match_mapping_type": "string",
              "path_match": "message"
            }
          },
          {
            "string_fields": {
              "mapping": {
                "type": "text",
                "norms": false,
                "fields": {
                  "keyword": {
                    "type": "keyword"
                  }
                }
              },
              "match_mapping_type": "string",
              "match": "*"
            }
          }
        ],
        "_all": {
          "norms": false,
          "enabled": true
        }
...
2

How are you telling this?

3

Thank you for you reply, I use '_cat/indices' to check the docs size between old and new index

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.