6.4 to 7.3, After Upgrade Elasticsearch uses 50% more Disk storage

oshan · February 11, 2020, 12:25pm

I am using ES 6.4 and after full indexing its size is 64GB for the Geo shapes index.
then I did all the code migrations and did the full indexing using ES 7.3 setup.
now that index size is 97GB. Is this an expected change? I can see Geo related changes on release notes. but this size is huge. any reasons?

Ignacio_Vera · February 11, 2020, 9:27pm

Hi,

In order to understand what is going on, could you share the mapping of your geo_shape field?

Thanks!

oshan · February 12, 2020, 12:24pm

Hi @Ignacio_Vera This gist contains mapping and settings of that index.
thanks for your time.

gist.github.com

https://gist.github.com/owexroasia/dced0004aaad91b026f251b9ce023ae0

mapping.json

{
  "assm": {
    "mappings": {
      "properties": {
        "assessors": {
          "type": "text",
          "store": true
        },
        "biogeolms": {
          "properties": {

This file has been truncated. show original

settings.json

{
  "assm": {
    "settings": {
      "index": {
        "mapping": {
          "nested_objects": {
            "limit": "125000"
          }
        },
        "number_of_shards": "5",

This file has been truncated. show original

Ignacio_Vera · February 14, 2020, 12:24pm

From your mappings, you are using new the indexing strategy based on vectorisation of shapes on top of Lucene BKD tree. In general we expect smaller index sizes, in particular when using prefix trees with high precision which is not your case.

Having a look into your mappings, are you using geo_shape field to index points? if so, have you consider using geo_point instead?

system · March 13, 2020, 12:24pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.