# Large number of primary shards for profiling indices

**URL:** <https://discuss.elastic.co/t/large-number-of-primary-shards-for-profiling-indices/367981>\
**Category:** APM\
**Tags:** server\
**Created:** [September 29, 2024, 9:13pm UTC](https://discuss.elastic.co/t/large-number-of-primary-shards-for-profiling-indices/367981 "2024-09-29T21:13:31Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![keiransteele](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keiransteele/32/93387_2.png) [@keiransteele](https://discuss.elastic.co/u/keiransteele)\
**Post date:** [September 29, 2024, 9:13pm UTC](https://discuss.elastic.co/t/large-number-of-primary-shards-for-profiling-indices/367981/1 "2024-09-29T21:13:31Z")

</div>

I'm wanting to find out why there are a large number of primary shards for profiling indices and if I can reduce the number of shards or remove the indices entirely given we are not using profiling?

We're on Elastic Cloud 8.15.2.

 ![2024-09-30_10-01](https://us1.discourse-cdn.com/elastic/original/3X/b/0/b09a4fef92573823cd044110ec6a67a51e2e01b3.jpeg)

These are the settings for the `.profiling-symbols-private-v001` index showing that `number_of_replicas` is `16`.

```auto
{
  "settings": {
    "index": {
      "routing": {
        "allocation": {
          "include": {
            "_tier_preference": "data_content"
          }
        }
      },
      "refresh_interval": "10s",
      "hidden": "true",
      "number_of_shards": "16",
      "auto_expand_replicas": "0-1",
      "blocks": {
        "read_only_allow_delete": "false"
      },
      "provided_name": ".profiling-symbols-private-v001",
      "creation_date": "1700511764812",
      "number_of_replicas": "1",
      "uuid": "zYvLWkcrQ_-wmpyt4WzoyQ",
      "version": {
        "created": "8090299"
      }
    }
  },
  "defaults": {
    "index": {
      "flush_after_merge": "512mb",
      "time_series": {
        "end_time": "9999-12-31T23:59:59.999Z",
        "start_time": "-9999-01-01T00:00:00Z",
        "es87tsdb_codec": {
          "enabled": "true"
        }
      },
      "final_pipeline": "_none",
      "max_inner_result_window": "100",
      "unassigned": {
        "node_left": {
          "delayed_timeout": "1m"
        }
      },
      "max_terms_count": "65536",
      "rollup": {
        "source": {
          "name": "",
          "uuid": ""
        }
      },
      "lifecycle": {
        "prefer_ilm": "true",
        "rollover_alias": "",
        "origination_date": "-1",
        "name": "",
        "parse_origination_date": "false",
        "step": {
          "wait_time_threshold": "12h"
        },
        "indexing_complete": "false"
      },
      "mode": "standard",
      "routing_partition_size": "1",
      "force_memory_term_dictionary": "false",
      "max_docvalue_fields_search": "100",
      "merge": {
        "scheduler": {
          "max_thread_count": "1",
          "auto_throttle": "true",
          "max_merge_count": "6"
        },
        "policy": {
          "merge_factor": "32",
          "floor_segment": "2mb",
          "max_merge_at_once_explicit": "30",
          "max_merge_at_once": "10",
          "max_merged_segment": "0b",
          "expunge_deletes_allowed": "10.0",
          "segments_per_tier": "10.0",
          "type": "UNSET",
          "deletes_pct_allowed": "20.0"
        }
      },
      "max_refresh_listeners": "1000",
      "max_regex_length": "1000",
      "load_fixed_bitset_filters_eagerly": "true",
      "number_of_routing_shards": "16",
      "write": {
        "wait_for_active_shards": "1"
      },
      "verified_before_close": "false",
      "mapping": {
        "coerce": "false",
        "nested_fields": {
          "limit": "50"
        },
        "depth": {
          "limit": "20"
        },
        "field_name_length": {
          "limit": "9223372036854775807"
        },
        "total_fields": {
          "limit": "1000",
          "ignore_dynamic_beyond_limit": "false"
        },
        "nested_objects": {
          "limit": "10000"
        },
        "ignore_malformed": "false",
        "dimension_fields": {
          "limit": "32768"
        },
        "synthetic_source": {
          "skip_ignored_source_read": "false",
          "skip_ignored_source_write": "false"
        }
      },
      "source_only": "false",
      "soft_deletes": {
        "enabled": "true",
        "retention": {
          "operations": "0"
        },
        "retention_lease": {
          "period": "12h"
        }
      },
      "max_script_fields": "32",
      "query": {
        "default_field": [
          "*"
        ],
        "parse": {
          "allow_unmapped_fields": "true"
        }
      },
      "format": "0",
      "frozen": "false",
      "sort": {
        "missing": [],
        "mode": [],
        "field": [],
        "order": []
      },
      "priority": "1",
      "routing_path": [],
      "version": {
        "compatibility": "8090299"
      },
      "codec": "default",
      "max_rescore_window": "10000",
      "bloom_filter_for_id_field": {
        "enabled": "true"
      },
      "max_adjacency_matrix_filters": "100",
      "analyze": {
        "max_token_count": "10000"
      },
      "gc_deletes": "60s",
      "top_metrics_max_size": "10",
      "failure_store": {
        "version": "0"
      },
      "optimize_auto_generated_id": "true",
      "max_ngram_diff": "1",
      "translog": {
        "flush_threshold_age": "1m",
        "generation_threshold_size": "64mb",
        "flush_threshold_size": "10gb",
        "sync_interval": "5s",
        "retention": {
          "size": "-1",
          "age": "-1"
        },
        "durability": "REQUEST"
      },
      "fast_refresh": "false",
      "mapper": {
        "dynamic": "true"
      },
      "recovery": {
        "type": ""
      },
      "requests": {
        "cache": {
          "enable": "true"
        }
      },
      "data_path": "",
      "highlight": {
        "max_analyzed_offset": "1000000",
        "weight_matches_mode": {
          "enabled": "true"
        }
      },
      "look_back_time": "2h",
      "routing": {
        "rebalance": {
          "enable": "all"
        },
        "allocation": {
          "disk": {
            "watermark": {
              "ignore": "false"
            }
          },
          "enable": "all",
          "total_shards_per_node": "-1"
        }
      },
      "search": {
        "slowlog": {
          "include": {
            "user": "false"
          },
          "level": "TRACE",
          "threshold": {
            "fetch": {
              "warn": "-1",
              "trace": "-1",
              "debug": "-1",
              "info": "-1"
            },
            "query": {
              "warn": "-1",
              "trace": "-1",
              "debug": "-1",
              "info": "-1"
            }
          }
        },
        "idle": {
          "after": "30s"
        },
        "throttled": "false"
      },
      "fielddata": {
        "cache": "node"
      },
      "look_ahead_time": "30m",
      "default_pipeline": "_none",
      "max_slices_per_scroll": "1024",
      "shard": {
        "check_on_startup": "false"
      },
      "xpack": {
        "watcher": {
          "template": {
            "version": ""
          }
        },
        "version": "",
        "ccr": {
          "following_index": "false"
        }
      },
      "percolator": {
        "map_unmapped_fields_as_text": "false"
      },
      "allocation": {
        "max_retries": "5",
        "existing_shards_allocator": "gateway_allocator"
      },
      "indexing": {
        "slowlog": {
          "include": {
            "user": "false"
          },
          "reformat": "true",
          "threshold": {
            "index": {
              "warn": "-1",
              "trace": "-1",
              "debug": "-1",
              "info": "-1"
            }
          },
          "source": "1000",
          "level": "TRACE"
        }
      },
      "compound_format": "1gb",
      "blocks": {
        "metadata": "false",
        "read": "false",
        "read_only": "false",
        "write": "false"
      },
      "max_result_window": "10000",
      "store": {
        "stats_refresh_interval": "10s",
        "type": "",
        "fs": {
          "fs_lock": "native"
        },
        "preload": [],
        "snapshot": {
          "snapshot_name": "",
          "index_uuid": "",
          "cache": {
            "prewarm": {
              "enabled": "true"
            },
            "enabled": "true",
            "excluded_file_types": []
          },
          "repository_uuid": "",
          "uncached_chunk_size": "-1b",
          "delete_searchable_snapshot": "false",
          "index_name": "",
          "partial": "false",
          "blob_cache": {
            "metadata_files": {
              "max_length": "64kb"
            }
          },
          "repository_name": "",
          "snapshot_uuid": ""
        }
      },
      "queries": {
        "cache": {
          "enabled": "true"
        }
      },
      "shard_limit": {
        "group": "normal"
      },
      "warmer": {
        "enabled": "true"
      },
      "downsample": {
        "interval": "",
        "source": {
          "name": "",
          "uuid": ""
        },
        "origin": {
          "name": "",
          "uuid": ""
        },
        "status": "unknown"
      },
      "override_write_load_forecast": "0.0",
      "max_shingle_diff": "3",
      "query_string": {
        "lenient": "false"
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Francesco\_Gualazzi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/francesco_gualazzi/32/126692_2.png) [@Francesco\_Gualazzi](https://discuss.elastic.co/u/Francesco_Gualazzi)\
**Post date:** [October 1, 2024, 2:56pm UTC](https://discuss.elastic.co/t/large-number-of-primary-shards-for-profiling-indices/367981/2 "2024-10-01T14:56:11Z")

</div>

Hello @keiransteele good to meet you.

You can definitely get rid of the indices, if they're allocating shards you want to free, but note that the indices are created empty and they don't take up disk space (in replicas) unless Profiling Agents are configured to ingest data.  
If your concern is disk space, you could as well leave them provisioned, and use Universal Profiling with the [OTel eBPF profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) to spot performance insights in your applications and detect costs across your services.  
If instead you are not going to adopt Profiling ever in this cluster, and you want to free up shards, I am sharing below how to do it.  
Before deleting the indices, know that they are managed by an Elasticsearch plugin that is not creating them unless Universal Profiling was enabled and set up on purpose via Kibana.

To disable Universal Profiling resource management, you can set a cluster setting using Kibana, after that you will be able to permanently delete the indices.

In Kibana, navigate to "Dev Tools" in the left menu, in the Console run the following query

```kibana
PUT /_cluster/settings
{
 "xpack.profiling.templates.enabled": false
}

```

Be aware this API request requires to be logged in as a user with elevated privileges, typically the `elastic` user.  
This will disable the Elasticsearch plugin resource management, after which you can issue a deletion of `.profiling-*` indices from the Stack Management UI.

Let me know if this solved your request.  
Cheers