Performance reduction in elasticsearch 8.0.0

Hello friends

Considering that Elasticsearch announced that it had upgraded its performance in version 8.0.0, I decided to install version 8.0.0, but after installation I found that my search performance was about 5 times slower than version 7.17 operates

I did not change the Elasticsearch settings in the previous version, nor in the current version, which is 8.0.0.

My data is the same as in version 7.17

It takes about 1.5 million data 100 milliseconds to get the answer back from Elasticsearch, while in version 7.17 it takes between 10 and 15 milliseconds.

Do you have a solution for this issue ???

You'll have to share a bunch more information for this to be replicated;

  • Your Elasticsearch config
  • Your mappings
  • Document samples
  • The requests you are making, and their responses
  • Ideally, Elasticsearch logs and the output from the _cluster/stats?pretty&human API?

Elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: amir
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
#network.host: 192.168.0.1
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

jvm.options

################################################################
##
## JVM configuration
##
################################################################
##
## WARNING: DO NOT EDIT THIS FILE. If you want to override the
## JVM options in this file, or set any additional options, you
## should create one or more files in the jvm.options.d
## directory containing your adjustments.
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/8.0/jvm-options.html
## for more information.
##
################################################################

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## The heap size is automatically configured by Elasticsearch
## based on the available memory in your system and the roles
## each node is configured to fulfill. If specifying heap is
## required, it should be done through a file in jvm.options.d,
## which should be named with .options suffix, and the min and
## max should be set to the same value. For example, to set the
## heap to 4 GB, create a new file in the jvm.options.d
## directory containing these lines:
##
-Xms512m
-Xmx512m
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/8.0/heap-size.html
## for more information
##
################################################################


################################################################
## Expert settings
################################################################
##
## All settings below here are considered expert settings. Do
## not adjust them unless you understand what you are doing. Do
## not edit them in this file; instead, create a new file in the
## jvm.options.d directory containing your adjustments.
##
################################################################

## GC configuration
8-13:-XX:+UseConcMarkSweepGC
8-13:-XX:CMSInitiatingOccupancyFraction=75
8-13:-XX:+UseCMSInitiatingOccupancyOnly

## G1GC Configuration
# to use G1GC, uncomment the next two lines and update the version on the
# following three lines to your version of the JDK
# 8-13:-XX:-UseConcMarkSweepGC
# 8-13:-XX:-UseCMSInitiatingOccupancyOnly
14-:-XX:+UseG1GC

## JVM temporary directory
-Djava.io.tmpdir=${ES_TMPDIR}

## heap dumps

# generate a heap dump when an allocation from the Java heap fails; heap dumps
# are created in the working directory of the JVM unless an alternative path is
# specified
-XX:+HeapDumpOnOutOfMemoryError

# exit right after heap dump on out of memory error. Recommended to also use
# on java 8 for supported versions (8u92+).
9-:-XX:+ExitOnOutOfMemoryError

# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
-XX:HeapDumpPath=/var/lib/elasticsearch

# specify an alternative path for JVM fatal error logs
-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log

## GC logging
-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m

document mapping

{
  "products" : {
    "mappings" : {
      "properties" : {
        "category" : {
          "properties" : {
            "id" : {
              "type" : "keyword"
            },
            "name" : {
              "type" : "text"
            }
          }
        },
        "city_id" : {
          "type" : "keyword"
        },
        "code" : {
          "type" : "keyword"
        },
        "comment_count" : {
          "type" : "integer"
        },
        "created_at" : {
          "type" : "date",
          "format" : "yyyy-MM-dd HH:mm"
        },
        "deleted_at" : {
          "type" : "date",
          "format" : "yyyy-MM-dd HH:mm",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "null_value" : "NULL"
            }
          }
        },
        "diversities" : {
          "type" : "nested",
          "properties" : {
            "deleted_at" : {
              "type" : "date",
              "format" : "yyyy-MM-dd HH:mm",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "null_value" : "NULL"
                }
              }
            },
            "expire" : {
              "type" : "date"
            },
            "expire_alert_level" : {
              "type" : "date"
            },
            "id" : {
              "type" : "keyword"
            },
            "inventory" : {
              "type" : "short"
            },
            "inventory_alert_level" : {
              "type" : "short"
            },
            "max_price" : {
              "type" : "unsigned_long"
            },
            "min_price" : {
              "type" : "unsigned_long"
            },
            "price_one" : {
              "type" : "unsigned_long"
            },
            "price_two" : {
              "type" : "unsigned_long",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "null_value" : "NULL"
                }
              }
            },
            "purchases_count" : {
              "type" : "integer"
            },
            "specifications" : {
              "type" : "nested",
              "properties" : {
                "id" : {
                  "type" : "keyword"
                },
                "tags" : {
                  "type" : "text",
                  "analyzer" : "standard_persian_analyzer"
                }
              }
            },
            "status" : {
              "type" : "boolean"
            },
            "unit" : {
              "properties" : {
                "id" : {
                  "type" : "keyword"
                },
                "name" : {
                  "type" : "text"
                }
              }
            }
          }
        },
        "favorite_count" : {
          "type" : "integer"
        },
        "id" : {
          "type" : "keyword"
        },
        "is_active" : {
          "type" : "boolean"
        },
        "location" : {
          "type" : "geo_point"
        },
        "name" : {
          "type" : "text",
          "fields" : {
            "_2gram" : {
              "type" : "text",
              "analyzer" : "2gram_persian_analyzer"
            },
            "edge_ngram" : {
              "type" : "text",
              "analyzer" : "edge_ngram_persian_analyzer"
            },
            "keyword" : {
              "type" : "text",
              "analyzer" : "keyword_persian_analyzer"
            },
            "standard" : {
              "type" : "text",
              "analyzer" : "standard_persian_analyzer"
            },
            "words_1_2" : {
              "type" : "text",
              "analyzer" : "words_1_2_persian_analyzer"
            },
            "words_3_4" : {
              "type" : "text",
              "analyzer" : "words_3_4_persian_analyzer"
            },
            "words_5_6" : {
              "type" : "text",
              "analyzer" : "words_5_6_persian_analyzer"
            }
          }
        },
        "purchases_count" : {
          "type" : "integer"
        },
        "score" : {
          "type" : "float"
        },
        "seller_id" : {
          "type" : "keyword"
        },
        "specifications" : {
          "type" : "nested",
          "properties" : {
            "id" : {
              "type" : "keyword"
            },
            "tags" : {
              "type" : "text",
              "analyzer" : "standard_persian_analyzer"
            }
          }
        },
        "status" : {
          "type" : "boolean"
        },
        "view_count" : {
          "type" : "integer"
        }
      }
    },
    "settings" : {
      "index" : {
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_content"
            }
          }
        },
        "number_of_shards" : "1",
        "provided_name" : "products",
        "creation_date" : "1644661979436",
        "analysis" : {
          "filter" : {
            "words_5_6_filter" : {
              "type" : "predicate_token_filter",
              "script" : {
                "source" : "token.getPosition() === 4 || token.getPosition() === 5"
              }
            },
            "persian_stem_multiplexer" : {
              "type" : "multiplexer",
              "filters" : [
                "parsi_stem_filter"
              ]
            },
            "piece" : {
              "type" : "edge_ngram",
              "min_gram" : "1",
              "max_gram" : "10"
            },
            "words_3_4_filter" : {
              "type" : "predicate_token_filter",
              "script" : {
                "source" : "token.getPosition() === 2 || token.getPosition() === 3"
              }
            },
            "2gram_filter" : {
              "max_shingle_size" : "2",
              "min_shingle_size" : "2",
              "output_unigrams" : "false",
              "type" : "shingle"
            },
            "words_1_2_filter" : {
              "type" : "predicate_token_filter",
              "script" : {
                "source" : "token.getPosition() === 0 || token.getPosition() === 1"
              }
            }
          },
          "analyzer" : {
            "2gram_persian_analyzer" : {
              "filter" : [
                "parsi_stop_filter",
                "decimal_digit",
                "parsi_normalizer",
                "persian_stem_multiplexer",
                "lowercase",
                "unique",
                "2gram_filter"
              ],
              "type" : "custom",
              "tokenizer" : "standard"
            },
            "words_1_2_persian_analyzer" : {
              "filter" : [
                "parsi_stop_filter",
                "decimal_digit",
                "parsi_normalizer",
                "persian_stem_multiplexer",
                "lowercase",
                "unique",
                "words_1_2_filter"
              ],
              "type" : "custom",
              "tokenizer" : "standard"
            },
            "edge_ngram_persian_analyzer" : {
              "filter" : [
                "parsi_stop_filter",
                "decimal_digit",
                "parsi_normalizer",
                "persian_stem_multiplexer",
                "lowercase",
                "piece",
                "unique"
              ],
              "type" : "custom",
              "tokenizer" : "standard"
            },
            "keyword_persian_analyzer" : {
              "filter" : [
                "parsi_stop_filter",
                "decimal_digit",
                "parsi_normalizer",
                "persian_stem_multiplexer",
                "lowercase"
              ],
              "type" : "custom",
              "tokenizer" : "keyword"
            },
            "standard_persian_analyzer" : {
              "filter" : [
                "parsi_stop_filter",
                "decimal_digit",
                "parsi_normalizer",
                "persian_stem_multiplexer",
                "lowercase",
                "unique"
              ],
              "type" : "custom",
              "tokenizer" : "standard"
            },
            "words_5_6_persian_analyzer" : {
              "filter" : [
                "parsi_stop_filter",
                "decimal_digit",
                "parsi_normalizer",
                "persian_stem_multiplexer",
                "lowercase",
                "unique",
                "words_5_6_filter"
              ],
              "type" : "custom",
              "tokenizer" : "standard"
            },
            "words_3_4_persian_analyzer" : {
              "filter" : [
                "parsi_stop_filter",
                "decimal_digit",
                "parsi_normalizer",
                "persian_stem_multiplexer",
                "lowercase",
                "unique",
                "words_3_4_filter"
              ],
              "type" : "custom",
              "tokenizer" : "standard"
            }
          }
        },
        "number_of_replicas" : "1",
        "uuid" : "qXpEQfFQRD-Q5BXyNQLyCA",
        "version" : {
          "created" : "8000099"
        }
      }
    }
  }
}

request

POST products/_search
{
  "size": 10,
  "from": 0,
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "seller_id": 1
          }
        },
        {
          "term": {
            "is_active": true
          }
        },
        {
          "term": {
            "deleted_at.keyword": "NULL"
          }
        },
        {
          "nested": {
            "path": "diversities",
            "query": {
              "bool": {
                "must": [
                  {
                    "term": {
                      "diversities.deleted_at.keyword": "NULL"
                    }
                  }
                ]
              }
            },
            "inner_hits": {
              "_source": [
                "diversities.inventory",
                "diversities.min_price",
                "diversities.price_one",
                "diversities.price_two",
                "diversities.unit.name"
              ]
            }
          }
        }
      ],
      "should": [
        {
          "match": {
            "name.standard": {
              "query": "ماشین لباسشویی",
              "fuzziness": "AUTO"
            }
          }
        },
        {
          "match": {
            "name.edge_ngram": {
              "query": "ماشین لباسشویی",
              "analyzer": "standard_persian_analyzer"
            }
          }
        },
        {
          "match": {
            "name._2gram": {
              "query": "ماشین لباسشویی",
              "analyzer": "standard_persian_analyzer",
              "fuzziness": 1,
              "fuzzy_transpositions": false
            }
          }
        },
        {
          "wildcard": {
            "name.keyword": {
              "value": "ماشین لباسشویی*",
              "boost": 45
            }
          }
        },
        {
          "match": {
            "name.words_1_2": {
              "query": "ماشین لباسشویی",
              "analyzer": "standard_persian_analyzer",
              "boost": 20
            }
          }
        },
        {
          "match": {
            "name.words_3_4": {
              "query": "ماشین لباسشویی",
              "analyzer": "standard_persian_analyzer",
              "boost": 10
            }
          }
        },
        {
          "match": {
            "name.words_5_6": {
              "query": "ماشین لباسشویی",
              "analyzer": "standard_persian_analyzer",
              "boost": 2.5
            }
          }
        },
        {
          "nested": {
            "path": "specifications",
            "query": {
              "bool": {
                "should": {
                  "match": {
                    "specifications.tags": "ماشین لباسشویی"
                  }
                }
              }
            }
          }
        },
        {
          "nested": {
            "path": "diversities",
            "query": {
              "bool": {
                "should": [
                  {
                    "nested": {
                      "path": "diversities.specifications",
                      "query": {
                        "bool": {
                          "should": {
                            "match": {
                              "diversities.specifications.tags": "ماشین لباسشویی"
                            }
                          }
                        }
                      }
                    }
                  }
                ]
              }
            }
          }
        }
      ]
    }
  },
  "_source": [
    "id",
    "name",
    "category.name"
  ],
  "sort": [
    "_score"
  ]
}

response

{
  "took" : 68,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : 393.95963,
    "hits" : [
      {
        "_index" : "products",
        "_id" : "_rCZ7X4BF3EMAgKsp2Fi",
        "_score" : 393.95963,
        "_source" : {
          "name" : "ماشین لباسشویی دوو مدل DWK-8543V ظرفیت 8 کیلوگرم",
          "id" : 90250,
          "category" : {
            "name" : "ماشین لباسشویی"
          }
        },
        "inner_hits" : {
          "diversities" : {
            "hits" : {
              "total" : {
                "value" : 5,
                "relation" : "eq"
              },
              "max_score" : 7.2332654E-7,
              "hits" : [
                {
                  "_index" : "products",
                  "_id" : "_rCZ7X4BF3EMAgKsp2Fi",
                  "_nested" : {
                    "field" : "diversities",
                    "offset" : 0
                  },
                  "_score" : 7.2332654E-7,
                  "_source" : {
                    "price_one" : 2468669,
                    "unit" : {
                      "name" : "باکس"
                    },
                    "min_price" : 8071341987224,
                    "price_two" : 17705588,
                    "inventory" : 4435
                  }
                },
                {
                  "_index" : "products",
                  "_id" : "_rCZ7X4BF3EMAgKsp2Fi",
                  "_nested" : {
                    "field" : "diversities",
                    "offset" : 1
                  },
                  "_score" : 7.2332654E-7,
                  "_source" : {
                    "price_one" : 65774174,
                    "unit" : {
                      "name" : "جعبه"
                    },
                    "min_price" : 4955091818947,
                    "price_two" : 42503230,
                    "inventory" : 9410
                  }
                },
                {
                  "_index" : "products",
                  "_id" : "_rCZ7X4BF3EMAgKsp2Fi",
                  "_nested" : {
                    "field" : "diversities",
                    "offset" : 2
                  },
                  "_score" : 7.2332654E-7,
                  "_source" : {
                    "price_one" : 28742346,
                    "unit" : {
                      "name" : "بسته"
                    },
                    "min_price" : 6020335509240,
                    "price_two" : 29187601,
                    "inventory" : 7062
                  }
                }
              ]
            }
          }
        }
      }
    ]
  }
}

_cluster/stats

{
  "_nodes" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "kJlPSZL3RRGOfKTdpg76ew",
  "timestamp" : 1644738131859,
  "status" : "yellow",
  "indices" : {
    "count" : 100,
    "shards" : {
      "total" : 101,
      "primaries" : 101,
      "replication" : 0.0,
      "index" : {
        "shards" : {
          "min" : 1,
          "max" : 2,
          "avg" : 1.01
        },
        "primaries" : {
          "min" : 1,
          "max" : 2,
          "avg" : 1.01
        },
        "replication" : {
          "min" : 0.0,
          "max" : 0.0,
          "avg" : 0.0
        }
      }
    },
    "docs" : {
      "count" : 4993395,
      "deleted" : 4942
    },
    "store" : {
      "size" : "1.2gb",
      "size_in_bytes" : 1291427968,
      "total_data_set_size" : "1.2gb",
      "total_data_set_size_in_bytes" : 1291427968,
      "reserved" : "0b",
      "reserved_in_bytes" : 0
    },
    "fielddata" : {
      "memory_size" : "0b",
      "memory_size_in_bytes" : 0,
      "evictions" : 0
    },
    "query_cache" : {
      "memory_size" : "13.4kb",
      "memory_size_in_bytes" : 13744,
      "total_count" : 2785,
      "hit_count" : 396,
      "miss_count" : 2389,
      "cache_size" : 11,
      "cache_count" : 11,
      "evictions" : 0
    },
    "completion" : {
      "size" : "0b",
      "size_in_bytes" : 0
    },
    "segments" : {
      "count" : 178,
      "memory" : "0b",
      "memory_in_bytes" : 0,
      "terms_memory" : "0b",
      "terms_memory_in_bytes" : 0,
      "stored_fields_memory" : "0b",
      "stored_fields_memory_in_bytes" : 0,
      "term_vectors_memory" : "0b",
      "term_vectors_memory_in_bytes" : 0,
      "norms_memory" : "0b",
      "norms_memory_in_bytes" : 0,
      "points_memory" : "0b",
      "points_memory_in_bytes" : 0,
      "doc_values_memory" : "0b",
      "doc_values_memory_in_bytes" : 0,
      "index_writer_memory" : "0b",
      "index_writer_memory_in_bytes" : 0,
      "version_map_memory" : "0b",
      "version_map_memory_in_bytes" : 0,
      "fixed_bit_set" : "1.1mb",
      "fixed_bit_set_memory_in_bytes" : 1249224,
      "max_unsafe_auto_id_timestamp" : 1644649168184,
      "file_sizes" : { }
    },
    "mappings" : {
      "field_types" : [
        {
          "name" : "alias",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "boolean",
          "count" : 170,
          "index_count" : 28,
          "script_count" : 0
        },
        {
          "name" : "constant_keyword",
          "count" : 9,
          "index_count" : 3,
          "script_count" : 0
        },
        {
          "name" : "date",
          "count" : 401,
          "index_count" : 69,
          "script_count" : 0
        },
        {
          "name" : "date_range",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "double",
          "count" : 12,
          "index_count" : 2,
          "script_count" : 0
        },
        {
          "name" : "float",
          "count" : 56,
          "index_count" : 15,
          "script_count" : 0
        },
        {
          "name" : "geo_point",
          "count" : 67,
          "index_count" : 12,
          "script_count" : 0
        },
        {
          "name" : "integer",
          "count" : 19,
          "index_count" : 15,
          "script_count" : 0
        },
        {
          "name" : "ip",
          "count" : 122,
          "index_count" : 13,
          "script_count" : 0
        },
        {
          "name" : "ip_range",
          "count" : 1,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "keyword",
          "count" : 5223,
          "index_count" : 85,
          "script_count" : 0
        },
        {
          "name" : "long",
          "count" : 583,
          "index_count" : 34,
          "script_count" : 0
        },
        {
          "name" : "nested",
          "count" : 8,
          "index_count" : 6,
          "script_count" : 0
        },
        {
          "name" : "object",
          "count" : 1330,
          "index_count" : 45,
          "script_count" : 0
        },
        {
          "name" : "short",
          "count" : 5,
          "index_count" : 4,
          "script_count" : 0
        },
        {
          "name" : "text",
          "count" : 537,
          "index_count" : 32,
          "script_count" : 0
        },
        {
          "name" : "unsigned_long",
          "count" : 4,
          "index_count" : 1,
          "script_count" : 0
        },
        {
          "name" : "version",
          "count" : 5,
          "index_count" : 5,
          "script_count" : 0
        }
      ],
      "runtime_field_types" : [ ]
    },
    "analysis" : {
      "char_filter_types" : [ ],
      "tokenizer_types" : [ ],
      "filter_types" : [
        {
          "name" : "edge_ngram",
          "count" : 2,
          "index_count" : 2
        },
        {
          "name" : "length",
          "count" : 1,
          "index_count" : 1
        },
        {
          "name" : "multiplexer",
          "count" : 1,
          "index_count" : 1
        },
        {
          "name" : "predicate_token_filter",
          "count" : 3,
          "index_count" : 1
        },
        {
          "name" : "shingle",
          "count" : 3,
          "index_count" : 2
        },
        {
          "name" : "stemmer",
          "count" : 1,
          "index_count" : 1
        },
        {
          "name" : "stop",
          "count" : 1,
          "index_count" : 1
        },
        {
          "name" : "word_delimiter_graph",
          "count" : 1,
          "index_count" : 1
        }
      ],
      "analyzer_types" : [
        {
          "name" : "custom",
          "count" : 14,
          "index_count" : 2
        }
      ],
      "built_in_char_filters" : [ ],
      "built_in_tokenizers" : [
        {
          "name" : "keyword",
          "count" : 1,
          "index_count" : 1
        },
        {
          "name" : "standard",
          "count" : 12,
          "index_count" : 2
        },
        {
          "name" : "whitespace",
          "count" : 1,
          "index_count" : 1
        }
      ],
      "built_in_filters" : [
        {
          "name" : "asciifolding",
          "count" : 7,
          "index_count" : 1
        },
        {
          "name" : "cjk_width",
          "count" : 7,
          "index_count" : 1
        },
        {
          "name" : "decimal_digit",
          "count" : 7,
          "index_count" : 1
        },
        {
          "name" : "lowercase",
          "count" : 14,
          "index_count" : 2
        },
        {
          "name" : "parsi_normalizer",
          "count" : 7,
          "index_count" : 1
        },
        {
          "name" : "parsi_stop_filter",
          "count" : 7,
          "index_count" : 1
        },
        {
          "name" : "unique",
          "count" : 6,
          "index_count" : 1
        }
      ],
      "built_in_analyzers" : [ ]
    },
    "versions" : [
      {
        "version" : "7.15.0",
        "index_count" : 16,
        "primary_shard_count" : 16,
        "total_primary_size" : "13.7mb",
        "total_primary_bytes" : 14463799
      },
      {
        "version" : "7.16.3",
        "index_count" : 73,
        "primary_shard_count" : 74,
        "total_primary_size" : "1mb",
        "total_primary_bytes" : 1107378
      },
      {
        "version" : "8.0.0",
        "index_count" : 11,
        "primary_shard_count" : 11,
        "total_primary_size" : "1.1gb",
        "total_primary_bytes" : 1275856791
      }
    ]
  },
  "nodes" : {
    "count" : {
      "total" : 1,
      "coordinating_only" : 0,
      "data" : 1,
      "data_cold" : 1,
      "data_content" : 1,
      "data_frozen" : 1,
      "data_hot" : 1,
      "data_warm" : 1,
      "ingest" : 1,
      "master" : 1,
      "ml" : 1,
      "remote_cluster_client" : 1,
      "transform" : 1,
      "voting_only" : 0
    },
    "versions" : [
      "8.0.0"
    ],
    "os" : {
      "available_processors" : 4,
      "allocated_processors" : 4,
      "names" : [
        {
          "name" : "Linux",
          "count" : 1
        }
      ],
      "pretty_names" : [
        {
          "pretty_name" : "Debian GNU/Linux 11 (bullseye)",
          "count" : 1
        }
      ],
      "architectures" : [
        {
          "arch" : "amd64",
          "count" : 1
        }
      ],
      "mem" : {
        "total" : "7.6gb",
        "total_in_bytes" : 8220798976,
        "adjusted_total" : "7.6gb",
        "adjusted_total_in_bytes" : 8220798976,
        "free" : "186.8mb",
        "free_in_bytes" : 195923968,
        "used" : "7.4gb",
        "used_in_bytes" : 8024875008,
        "free_percent" : 2,
        "used_percent" : 98
      }
    },
    "process" : {
      "cpu" : {
        "percent" : 0
      },
      "open_file_descriptors" : {
        "min" : 827,
        "max" : 827,
        "avg" : 827
      }
    },
    "jvm" : {
      "max_uptime" : "1.4h",
      "max_uptime_in_millis" : 5385499,
      "versions" : [
        {
          "version" : "17.0.1",
          "vm_name" : "OpenJDK 64-Bit Server VM",
          "vm_version" : "17.0.1+12",
          "vm_vendor" : "Eclipse Adoptium",
          "bundled_jdk" : true,
          "using_bundled_jdk" : true,
          "count" : 1
        }
      ],
      "mem" : {
        "heap_used" : "405.4mb",
        "heap_used_in_bytes" : 425099272,
        "heap_max" : "512mb",
        "heap_max_in_bytes" : 536870912
      },
      "threads" : 64
    },
    "fs" : {
      "total" : "119.5gb",
      "total_in_bytes" : 128336859136,
      "free" : "72.8gb",
      "free_in_bytes" : 78178045952,
      "available" : "66.6gb",
      "available_in_bytes" : 71611445248
    },
    "plugins" : [
      {
        "name" : "ParsiAnalyzer",
        "version" : "1.0",
        "elasticsearch_version" : "8.0.0",
        "java_version" : "1.8",
        "description" : "Persian analyzer for elasticsearch.",
        "classname" : "org.elasticsearch.analyzer.ParsiAnalyzerPlugin",
        "extended_plugins" : [ ],
        "has_native_controller" : false,
        "licensed" : false,
        "type" : "isolated"
      }
    ],
    "network_types" : {
      "transport_types" : {
        "security4" : 1
      },
      "http_types" : {
        "security4" : 1
      }
    },
    "discovery_types" : {
      "zen" : 1
    },
    "packaging_types" : [
      {
        "flavor" : "default",
        "type" : "deb",
        "count" : 1
      }
    ],
    "ingest" : {
      "number_of_pipelines" : 2,
      "processor_stats" : {
        "gsub" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "remove" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "script" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "set" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "set_security_user" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        }
      }
    }
  }
}

I think that you have a lot of small indices. Specifically for the tiny heap you have.
I'd not set the heap size and let Elasticsearch decide or at least I'd use half of the RAM to it.

I removed the extra indices but there was no change in performance (but according to you, the number of my indices was high)

The heap memory in version 7.16 was exactly the same as 512mg

Changing the heap memory does not change the speed

Do you have another solution ??

Is that happening on the first run? Is there any other service running on your machine?
What is now the output of the cluster stats request?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.