# Problem elastic. uneven distribution of the data node **URL:** \ **Category:** Elasticsearch\ **Created:** [February 2, 2024, 1:54pm UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409 "2024-02-02T13:54:38Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![San9](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@San9](https://discuss.elastic.co/u/San9)\ **Post date:** [February 2, 2024, 1:54pm UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/1 "2024-02-02T13:54:38Z")
Hi Team! I'm using the elastic version 8.11.1. Recently I began to notice an uneven distribution of indexes across data nodes. ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/e/ce4aa5e86e8ff7a00e5bf1e5d42ce9fb81a45700.png) Alert: Node v-elk-ed02 is reporting disk usage of 90% at February 2, 2024 3:44 PM EET utilization ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/8/b8472f85c4ce07b87298db66967526aa1d6dea40.png) I mainly use 4 or 2 shards for indexes. Status cluster - green. What is needed for data to be distributed evenly across all data nodes? Thanks from your help. ---
**Author:** ![hendry.lim](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendry.lim/32/71328_2.png) [@hendry.lim](https://discuss.elastic.co/u/hendry.lim)\ **Post date:** [February 2, 2024, 2:34pm UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/2 "2024-02-02T14:34:46Z")
You can set `cluster.routing.allocation.balance.disk_usage` to a higher value to make it a higher priority. But, something to take note is that the shard movement will be more aggressive with the disk usage set as a higher priority. > **[Cluster-level shard allocation and routing settings | Elasticsearch Guide...](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cluster.html#shards-rebalancing-heuristics)** ---
**Author:** ![San9](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@San9](https://discuss.elastic.co/u/San9)\ **Post date:** [February 2, 2024, 4:04pm UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/3 "2024-02-02T16:04:47Z")
Thanks, how can I see the priority that is currently used for the cluster nodes? I also noticed that some shards with one index are located only on this node (all 4 are on one, but other shard nodes do not have this index). ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/2/c2ffbfc4c265ac09e6c25916fa02f9f4f72a6f8b.png) This is an index without replicas, only 4 shards. And not only him. ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/e/6ed18813e88001fe766a05537914ff1f67f0bcea.png) ---
**Author:** ![hendry.lim](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendry.lim/32/71328_2.png) [@hendry.lim](https://discuss.elastic.co/u/hendry.lim)\ **Post date:** [February 2, 2024, 11:27pm UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/4 "2024-02-02T23:27:38Z")
The shard allocation heuristics setting priorities are mentioned in the doc I linked above. You also can use the `GET _cluster/settings?flat_settings&include_defaults` API to get the all cluster settings including the default ones. ---
**Author:** ![San9](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@San9](https://discuss.elastic.co/u/San9)\ **Post date:** [February 5, 2024, 2:07pm UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/5 "2024-02-05T14:07:17Z")
Thanks from information. Here is my command output, this is my first time encountering this and I don’t know what to do with it or where to start. GET \_cluster/settings?flat\_settings&include\_defaults ```auto { "persistent": { "xpack.monitoring.collection.enabled": "true" }, "transient": {}, "defaults": { "action.auto_create_index": "true", "action.destructive_requires_name": "true", "action.search.pre_filter_shard_size.default": "128", "action.search.shard_count.limit": "9223372036854775807", "async_search.index_cleanup_interval": "1h", "bootstrap.ctrlhandler": "true", "bootstrap.memory_lock": "true", "cache.recycler.page.limit.heap": "10%", "cache.recycler.page.type": "CONCURRENT", "cache.recycler.page.weight.bytes": "1.0", "cache.recycler.page.weight.ints": "1.0", "cache.recycler.page.weight.longs": "1.0", "cache.recycler.page.weight.objects": "0.1", "ccr.auto_follow.wait_for_metadata_timeout": "60s", "ccr.indices.recovery.chunk_size": "1mb", "ccr.indices.recovery.internal_action_timeout": "60s", "ccr.indices.recovery.max_bytes_per_sec": "40mb", "ccr.indices.recovery.max_concurrent_file_chunks": "5", "ccr.indices.recovery.recovery_activity_timeout": "60s", "ccr.wait_for_metadata_timeout": "60s", "client.type": "node", "cluster.auto_shrink_voting_configuration": "true", "cluster.blocks.read_only": "false", "cluster.blocks.read_only_allow_delete": "false", "cluster.deprecation_indexing.enabled": "true", "cluster.deprecation_indexing.x_opaque_id_used.enabled": "true", "cluster.discovery_configuration_check.interval": "30000ms", "cluster.election.back_off_time": "100ms", "cluster.election.duration": "500ms", "cluster.election.initial_timeout": "100ms", "cluster.election.max_timeout": "10s", "cluster.election.strategy": "supports_voting_only", "cluster.fault_detection.follower_check.interval": "1000ms", "cluster.fault_detection.follower_check.retry_count": "3", "cluster.fault_detection.follower_check.timeout": "10000ms", "cluster.fault_detection.leader_check.interval": "1000ms", "cluster.fault_detection.leader_check.retry_count": "3", "cluster.fault_detection.leader_check.timeout": "10000ms", "cluster.follower_lag.timeout": "90000ms", "cluster.indices.close.enable": "true", "cluster.indices.tombstones.size": "500", "cluster.info.update.interval": "30s", "cluster.info.update.timeout": "15s", "cluster.initial_master_nodes": [], "cluster.join_validation.cache_timeout": "60s", "cluster.lifecycle.default.rollover": "max_age=auto,max_primary_shard_size=50gb,min_docs=1,max_primary_shard_docs=200000000", "cluster.max_shards_per_node": "1000", "cluster.max_shards_per_node.frozen": "3000", "cluster.max_voting_config_exclusions": "10", "cluster.name": "my-elk-prod", "cluster.no_master_block": "write", "cluster.nodes.reconnect_interval": "10s", "cluster.persistent_tasks.allocation.enable": "all", "cluster.persistent_tasks.allocation.recheck_interval": "30s", "cluster.publish.info_timeout": "10000ms", "cluster.publish.timeout": "30000ms", "cluster.remote.connections_per_cluster": "3", "cluster.remote.initial_connect_timeout": "30s", "cluster.remote.node.attr": "", "cluster.routing.allocation.allow_rebalance": "indices_all_active", "cluster.routing.allocation.awareness.attributes": [], "cluster.routing.allocation.balance.disk_usage": "2.0E-11", "cluster.routing.allocation.balance.index": "0.55", "cluster.routing.allocation.balance.shard": "0.45", "cluster.routing.allocation.balance.threshold": "1.0", "cluster.routing.allocation.balance.write_load": "10.0", "cluster.routing.allocation.cluster_concurrent_rebalance": "2", "cluster.routing.allocation.desired_balance.progress_log_interval": "1m", "cluster.routing.allocation.desired_balance.undesired_allocations.log_interval": "1h", "cluster.routing.allocation.desired_balance.undesired_allocations.threshold": "0.1", "cluster.routing.allocation.disk.reroute_interval": "60s", "cluster.routing.allocation.disk.threshold_enabled": "true", "cluster.routing.allocation.disk.watermark.enable_for_single_data_node": "true", "cluster.routing.allocation.disk.watermark.flood_stage": "95%", "cluster.routing.allocation.disk.watermark.flood_stage.frozen": "95%", "cluster.routing.allocation.disk.watermark.flood_stage.frozen.max_headroom": "20GB", "cluster.routing.allocation.disk.watermark.flood_stage.max_headroom": "100GB", "cluster.routing.allocation.disk.watermark.high": "90%", "cluster.routing.allocation.disk.watermark.high.max_headroom": "150GB", "cluster.routing.allocation.disk.watermark.low": "85%", "cluster.routing.allocation.disk.watermark.low.max_headroom": "200GB", "cluster.routing.allocation.enable": "all", "cluster.routing.allocation.enforce_default_tier_preference": "true", "cluster.routing.allocation.node_concurrent_incoming_recoveries": "2", "cluster.routing.allocation.node_concurrent_outgoing_recoveries": "2", "cluster.routing.allocation.node_concurrent_recoveries": "2", "cluster.routing.allocation.node_initial_primaries_recoveries": "4", "cluster.routing.allocation.same_shard.host": "false", "cluster.routing.allocation.total_shards_per_node": "-1", "cluster.routing.allocation.type": "desired_balance", "cluster.routing.rebalance.enable": "all", "cluster.routing.use_adaptive_replica_selection": "true", "cluster.service.master_service_starvation_logging_threshold": "5m", "cluster.service.slow_master_task_logging_threshold": "10s", "cluster.service.slow_task_logging_threshold": "30s", "cluster.snapshot.info.max_concurrent_fetches": "5", "cluster_state.document_page_size": "1mb", "data_streams.lifecycle.poll_interval": "5m", "data_streams.lifecycle.target.merge.policy.floor_segment": "100mb", "data_streams.lifecycle.target.merge.policy.merge_factor": "16", "deprecation.skip_deprecated_settings": [], "discovery.cluster_formation_warning_timeout": "10000ms", "discovery.find_peers_interval": "1000ms", "discovery.initial_state_timeout": "30s", "discovery.probe.connect_timeout": "30s", "discovery.probe.handshake_timeout": "30s", "discovery.request_peers_timeout": "3000ms", "discovery.seed_hosts": [ "v-elk-em01.my.cluster:9301", "v-elk-em02.my.cluster:9301", "v-elk-em03.my.cluster:9301" ], "discovery.seed_providers": [], "discovery.seed_resolver.max_concurrent_resolvers": "10", "discovery.seed_resolver.timeout": "5s", "discovery.type": "multi-node", "discovery.unconfigured_bootstrap_timeout": "3s", "enrich.cache_size": "1000", "enrich.cleanup_period": "15m", "enrich.coordinator_proxy.max_concurrent_requests": "8", "enrich.coordinator_proxy.max_lookups_per_request": "128", "enrich.coordinator_proxy.queue_capacity": "1024", "enrich.fetch_size": "10000", "enrich.max_concurrent_policy_executions": "50", "enrich.max_force_merge_attempts": "3", "esql.query.result_truncation_default_size": "500", "esql.query.result_truncation_max_size": "10000", "esql.queue_size": "1000", "esql.size": "4", "esql_worker.queue_size": "1000", "esql_worker.size": "7", "gateway.expected_data_nodes": "-1", "gateway.recover_after_data_nodes": "-1", "gateway.recover_after_time": "0ms", "gateway.slow_write_logging_threshold": "10s", "gateway.write_dangling_indices_info": "true", "health.ilm.max_retries_per_step": "100", "health.ilm.max_time_on_action": "1d", "health.ilm.max_time_on_step": "1d", "health.master_history.has_master_lookup_timeframe": "30s", "health.master_history.identity_changes_threshold": "4", "health.master_history.no_master_transitions_threshold": "4", "health.node.enabled": "true", "health.periodic_logger.enabled": "false", "health.periodic_logger.poll_interval": "60s", "health.reporting.local.monitor.interval": "30s", "health_node.transport_action_timeout": "5s", "http.bind_host": [ "0.0.0.0" ], "http.client_stats.closed_channels.max_age": "5m", "http.client_stats.closed_channels.max_count": "10000", "http.client_stats.enabled": "true", "http.compression": "false", "http.compression_level": "3", "http.cors.allow-credentials": "false", "http.cors.allow-headers": "X-Requested-With,Content-Type,Content-Length,Authorization,Accept,User-Agent,X-Elastic-Client-Meta", "http.cors.allow-methods": "OPTIONS,HEAD,GET,POST,PUT,DELETE", "http.cors.allow-origin": "", "http.cors.enabled": "false", "http.cors.max-age": "1728000", "http.detailed_errors.enabled": "true", "http.host": [ "0.0.0.0" ], "http.max_chunk_size": "8kb", "http.max_content_length": "100mb", "http.max_header_size": "16kb", "http.max_initial_line_length": "4kb", "http.max_warning_header_count": "-1", "http.max_warning_header_size": "-1b", "http.netty.max_composite_buffer_components": "69905", "http.netty.receive_predictor_size": "64kb", "http.netty.worker_count": "0", "http.pipelining.max_events": "10000", "http.port": "9200", "http.publish_host": [ "0.0.0.0" ], "http.publish_port": "-1", "http.read_timeout": "0ms", "http.reset_cookies": "false", "http.shutdown_grace_period": "0ms", "http.tcp.keep_alive": "true", "http.tcp.keep_count": "-1", "http.tcp.keep_idle": "-1", "http.tcp.keep_interval": "-1", "http.tcp.no_delay": "true", "http.tcp.receive_buffer_size": "-1b", "http.tcp.reuse_address": "true", "http.tcp.send_buffer_size": "-1b", "http.tracer.exclude": [], "http.tracer.include": [], "http.type": "security4", "http.type.default": "netty4", "index.codec": "default", "index.recovery.type": "", "index.store.fs.fs_lock": "native", "index.store.preload": [], "index.store.snapshot.cache.excluded_file_types": [], "index.store.snapshot.uncached_chunk_size": "-1b", "index.store.type": "", "indexing_pressure.memory.limit": "10%", "indices.analysis.hunspell.dictionary.ignore_case": "false", "indices.analysis.hunspell.dictionary.lazy": "false", "indices.breaker.fielddata.limit": "40%", "indices.breaker.fielddata.overhead": "1.03", "indices.breaker.fielddata.type": "memory", "indices.breaker.request.limit": "60%", "indices.breaker.request.overhead": "1.0", "indices.breaker.request.type": "memory", "indices.breaker.total.limit": "95%", "indices.breaker.total.use_real_memory": "true", "indices.breaker.type": "hierarchy", "indices.cache.cleanup_interval": "1m", "indices.fielddata.cache.size": "-1b", "indices.id_field_data.enabled": "false", "indices.lifecycle.history_index_enabled": "true", "indices.lifecycle.poll_interval": "10m", "indices.lifecycle.rollover.only_if_has_documents": "true", "indices.lifecycle.step.master_timeout": "30s", "indices.mapping.dynamic_timeout": "30s", "indices.mapping.max_in_flight_updates": "10", "indices.memory.index_buffer_size": "10%", "indices.memory.interval": "5s", "indices.memory.max_index_buffer_size": "-1", "indices.memory.min_index_buffer_size": "48mb", "indices.memory.shard_inactive_time": "5m", "indices.queries.cache.all_segments": "false", "indices.queries.cache.count": "10000", "indices.queries.cache.size": "10%", "indices.query.bool.max_clause_count": "4096", "indices.query.bool.max_nested_depth": "30", "indices.query.query_string.allowLeadingWildcard": "true", "indices.query.query_string.analyze_wildcard": "false", "indices.recovery.internal_action_long_timeout": "1800000ms", "indices.recovery.internal_action_retry_timeout": "1m", "indices.recovery.internal_action_timeout": "15m", "indices.recovery.max_bytes_per_sec": "40mb", "indices.recovery.max_concurrent_file_chunks": "2", "indices.recovery.max_concurrent_operations": "1", "indices.recovery.max_concurrent_snapshot_file_downloads": "5", "indices.recovery.max_concurrent_snapshot_file_downloads_per_node": "25", "indices.recovery.recovery_activity_timeout": "1800000ms", "indices.recovery.retry_delay_network": "5s", "indices.recovery.retry_delay_state_sync": "500ms", "indices.recovery.use_snapshots": "true", "indices.replication.initial_retry_backoff_bound": "50ms", "indices.replication.retry_timeout": "60s", "indices.requests.cache.expire": "0ms", "indices.requests.cache.size": "1%", "indices.store.delete.shard.timeout": "30s", "indices.store.shard_lock_retry.interval": "1s", "indices.store.shard_lock_retry.timeout": "1m", "indices.write_ack_delay_interval": "0ms", "indices.write_ack_delay_randomness_bound": "70ms", "ingest.geoip.cache_size": "1000", "ingest.geoip.downloader.eager.download": "false", "ingest.geoip.downloader.enabled": "true", "ingest.geoip.downloader.endpoint": "https://geoip.elastic.co/v1/database", "ingest.geoip.downloader.poll.interval": "3d", "ingest.grok.watchdog.interval": "1s", "ingest.grok.watchdog.max_execution_time": "1s", "ingest.user_agent.cache_size": "1000", "ingest_node.transport_action_timeout": "20s", "logger.level": "INFO", "master_history.max_age": "30m", "monitor.fs.health.enabled": "true", "monitor.fs.health.refresh_interval": "120s", "monitor.fs.health.slow_path_logging_threshold": "5s", "monitor.fs.refresh_interval": "1s", "monitor.jvm.gc.enabled": "true", "monitor.jvm.gc.overhead.debug": "10", "monitor.jvm.gc.overhead.info": "25", "monitor.jvm.gc.overhead.warn": "50", "monitor.jvm.gc.refresh_interval": "1s", "monitor.jvm.refresh_interval": "1s", "monitor.os.refresh_interval": "1s", "monitor.process.refresh_interval": "1s", "network.bind_host": [ "0" ], "network.breaker.inflight_requests.limit": "100%", "network.breaker.inflight_requests.overhead": "2.0", "network.host": [ "0" ], "network.publish_host": [ "0" ], "network.server": "true", "network.tcp.keep_alive": "true", "network.tcp.keep_count": "-1", "network.tcp.keep_idle": "-1", "network.tcp.keep_interval": "-1", "network.tcp.no_delay": "true", "network.tcp.receive_buffer_size": "-1b", "network.tcp.reuse_address": "true", "network.tcp.send_buffer_size": "-1b", "node._internal.default_refresh_interval": "1s", "node.attr.ml.config_version": "11.0.0", "node.attr.transform.config_version": "10.0.0", "node.attr.xpack.installed": "true", "node.bandwidth.recovery.disk.read": "-1", "node.bandwidth.recovery.disk.write": "-1", "node.bandwidth.recovery.factor.read": "0.4", "node.bandwidth.recovery.factor.write": "0.4", "node.bandwidth.recovery.network": "-1", "node.bandwidth.recovery.operator.factor": "0.4", "node.bandwidth.recovery.operator.factor.max_overcommit": "100.0", "node.bandwidth.recovery.operator.factor.read": "0.4", "node.bandwidth.recovery.operator.factor.write": "0.4", "node.enable_lucene_segment_infos_trace": "false", "node.external_id": "", "node.id.seed": "0", "node.name": "v-elk-ekb01.my.cluster", "node.portsfile": "false", "node.processors": "4.0", "node.roles": [], "node.store.allow_mmap": "true", "path.data": [ "/var/lib/elasticsearch" ], "path.home": "/usr/share/elasticsearch", "path.logs": "/var/log/elasticsearch", "path.repo": [ "/mnt/nfs-share" ], "path.shared_data": "", "plugin.mandatory": [], "readiness.port": "-1", "reindex.remote.whitelist": [], "remote_cluster.bind_host": [ "0.0.0.0" ], "remote_cluster.host": [ "0.0.0.0" ], "remote_cluster.max_request_header_size": "64kb", "remote_cluster.port": "9443", "remote_cluster.publish_host": [ "0.0.0.0" ], "remote_cluster.publish_port": "-1", "remote_cluster.tcp.keep_alive": "true", "remote_cluster.tcp.keep_count": "-1", "remote_cluster.tcp.keep_idle": "-1", "remote_cluster.tcp.keep_interval": "-1", "remote_cluster.tcp.no_delay": "true", "remote_cluster.tcp.reuse_address": "true", "remote_cluster.tcp.send_buffer_size": "-1b", "remote_cluster_server.enabled": "false", "repositories.fs.chunk_size": "9223372036854775807b", "repositories.fs.location": "", "repositories.url.allowed_urls": [], "repositories.url.supported_protocols": [ "http", "https", "ftp", "file", "jar" ], "repositories.url.url": "http:", "repository_s3.compare_and_exchange.anti_contention_delay": "1s", "repository_s3.compare_and_exchange.time_to_live": "15s", "resource.reload.enabled": "true", "resource.reload.interval.high": "5s", "resource.reload.interval.low": "60s", "resource.reload.interval.medium": "30s", "rest.action.multi.allow_explicit_index": "true", "runtime_fields.grok.watchdog.interval": "1s", "runtime_fields.grok.watchdog.max_execution_time": "1s", "script.allowed_contexts": [], "script.allowed_types": [], "script.cache.expire": "0ms", "script.cache.max_size": "3000", "script.disable_max_compilations_rate": "false", "script.max_compilations_rate": "150/5m", "script.max_size_in_bytes": "65535", "script.painless.regex.enabled": "limited", "script.painless.regex.limit-factor": "6", "search.aggs.rewrite_to_filter_by_filter": "true", "search.aggs.tdigest_execution_hint": "DEFAULT", "search.allow_expensive_queries": "true", "search.check_ccs_compatibility": "false", "search.default_allow_partial_results": "true", "search.default_keep_alive": "5m", "search.default_search_timeout": "-1", "search.highlight.term_vector_multi_value": "true", "search.keep_alive_interval": "1m", "search.low_level_cancellation": "true", "search.max_async_search_response_size": "10mb", "search.max_buckets": "65536", "search.max_keep_alive": "24h", "search.max_open_scroll_context": "500", "search.query_phase_parallel_collection_enabled": "false", "search.worker_threads_enabled": "true", "searchable_snapshots.blob_cache.periodic_cleanup.batch_size": "100", "searchable_snapshots.blob_cache.periodic_cleanup.interval": "1h", "searchable_snapshots.blob_cache.periodic_cleanup.pit_keep_alive": "10m", "searchable_snapshots.blob_cache.periodic_cleanup.retention_period": "1h", "security.manager.filter_bad_defaults": "true", "slm.health.failed_snapshot_warn_threshold": "5", "slm.history_index_enabled": "true", "slm.minimum_interval": "15m", "slm.retention_duration": "1h", "slm.retention_schedule": "0 30 1 * * ?", "snapshot.max_concurrent_operations": "1000", "snapshot.refresh_repo_uuid_on_restore": "true", "stack.templates.enabled": "true", "telemetry.metrics.enabled": "false", "thread_pool.analyze.queue_size": "16", "thread_pool.analyze.size": "1", "thread_pool.auto_complete.queue_size": "100", "thread_pool.auto_complete.size": "1", "thread_pool.azure_event_loop.core": "0", "thread_pool.azure_event_loop.keep_alive": "30s", "thread_pool.azure_event_loop.max": "1", "thread_pool.cluster_coordination.queue_size": "-1", "thread_pool.cluster_coordination.size": "1", "thread_pool.estimated_time_interval": "200ms", "thread_pool.estimated_time_interval.warn_threshold": "5s", "thread_pool.fetch_shard_started.core": "1", "thread_pool.fetch_shard_started.keep_alive": "5m", "thread_pool.fetch_shard_started.max": "8", "thread_pool.fetch_shard_store.core": "1", "thread_pool.fetch_shard_store.keep_alive": "5m", "thread_pool.fetch_shard_store.max": "8", "thread_pool.flush.core": "1", "thread_pool.flush.keep_alive": "5m", "thread_pool.flush.max": "2", "thread_pool.force_merge.queue_size": "-1", "thread_pool.force_merge.size": "1", "thread_pool.generic.core": "4", "thread_pool.generic.keep_alive": "30s", "thread_pool.generic.max": "128", "thread_pool.get.queue_size": "1000", "thread_pool.get.size": "7", "thread_pool.management.core": "1", "thread_pool.management.keep_alive": "5m", "thread_pool.management.max": "4", "thread_pool.profiling.core": "0", "thread_pool.profiling.keep_alive": "30m", "thread_pool.profiling.max": "1", "thread_pool.refresh.core": "1", "thread_pool.refresh.keep_alive": "5m", "thread_pool.refresh.max": "2", "thread_pool.repository_azure.core": "0", "thread_pool.repository_azure.keep_alive": "30s", "thread_pool.repository_azure.max": "5", "thread_pool.scheduler.warn_threshold": "5s", "thread_pool.search.queue_size": "1000", "thread_pool.search.size": "7", "thread_pool.search_coordination.queue_size": "1000", "thread_pool.search_coordination.size": "2", "thread_pool.search_throttled.queue_size": "100", "thread_pool.search_throttled.size": "1", "thread_pool.search_worker.queue_size": "-1", "thread_pool.search_worker.size": "7", "thread_pool.snapshot.core": "1", "thread_pool.snapshot.keep_alive": "5m", "thread_pool.snapshot.max": "10", "thread_pool.snapshot_meta.core": "1", "thread_pool.snapshot_meta.keep_alive": "30s", "thread_pool.snapshot_meta.max": "12", "thread_pool.system_critical_read.queue_size": "2000", "thread_pool.system_critical_read.size": "2", "thread_pool.system_critical_write.queue_size": "1500", "thread_pool.system_critical_write.size": "2", "thread_pool.system_read.queue_size": "2000", "thread_pool.system_read.size": "2", "thread_pool.system_write.queue_size": "1000", "thread_pool.system_write.size": "2", "thread_pool.warmer.core": "1", "thread_pool.warmer.keep_alive": "5m", "thread_pool.warmer.max": "2", "thread_pool.write.queue_size": "10000", "thread_pool.write.size": "4", "time_series.poll_interval": "5m", "tracing.apm.enabled": "false", "tracing.apm.names.exclude": [], "tracing.apm.names.include": [], "tracing.apm.sanitize_field_names": [ "password", "passwd", "pwd", "secret", "*key", "*token*", "*session*", "*credit*", "*card*", "*auth*", "*principal*", "set-cookie" ], "transport.bind_host": [ "0.0.0.0" ], "transport.compress": "INDEXING_DATA", "transport.compression_scheme": "LZ4", "transport.connect_timeout": "30s", "transport.connections_per_node.bulk": "3", "transport.connections_per_node.ping": "1", "transport.connections_per_node.recovery": "2", "transport.connections_per_node.reg": "6", "transport.connections_per_node.state": "1", "transport.host": [ "0.0.0.0" ], "transport.netty.boss_count": "1", "transport.netty.receive_predictor_max": "64kb", "transport.netty.receive_predictor_min": "64kb", "transport.netty.receive_predictor_size": "64kb", "transport.netty.worker_count": "4", "transport.ping_schedule": "-1", "transport.port": "9301", "transport.publish_host": [ "0.0.0.0" ], "transport.publish_port": "-1", "transport.rst_on_close": "false", "transport.slow_operation_logging_threshold": "5s", "transport.tcp.keep_alive": "true", "transport.tcp.keep_count": "-1", "transport.tcp.keep_idle": "-1", "transport.tcp.keep_interval": "-1", "transport.tcp.no_delay": "true", "transport.tcp.receive_buffer_size": "-1b", "transport.tcp.reuse_address": "true", "transport.tcp.send_buffer_size": "-1b", "transport.tracer.exclude": [ "internal:coordination/fault_detection/*" ], "transport.tracer.include": [], "transport.type": "security4", "transport.type.default": "netty4", "write_load_forecaster.max_index_age": "7d", "xpack.applications.behavioral_analytics.ingest.bulk_processor.flush_delay": "10s", "xpack.applications.behavioral_analytics.ingest.bulk_processor.max_bytes_in_flight": "5%", "xpack.applications.behavioral_analytics.ingest.bulk_processor.max_events_per_bulk": "500", "xpack.applications.behavioral_analytics.ingest.bulk_processor.max_number_of_retries": "1", "xpack.applications.rules.max_rules_per_ruleset": "100", "xpack.autoscaling.memory.monitor.timeout": "15s", "xpack.ccr.ccr_thread_pool.queue_size": "100", "xpack.ccr.ccr_thread_pool.size": "32", "xpack.ccr.enabled": "true", "xpack.downsample.thread_pool.queue_size": "256", "xpack.downsample.thread_pool.size": "1", "xpack.ent_search.enabled": "true", "xpack.eql.enabled": "true", "xpack.graph.enabled": "true", "xpack.http.connection_pool_ttl": "-1", "xpack.http.default_connection_timeout": "10s", "xpack.http.default_read_timeout": "10s", "xpack.http.max_response_size": "10mb", "xpack.http.proxy.host": "", "xpack.http.proxy.port": "0", "xpack.http.proxy.scheme": "", "xpack.http.tcp.keep_alive": "true", "xpack.http.whitelist": [ "*" ], "xpack.idp.allowed_nameid_formats": [ "urn:oasis:names:tc:SAML:2.0:nameid-format:transient" ], "xpack.idp.contact.email": "", "xpack.idp.contact.given_name": "", "xpack.idp.contact.surname": "", "xpack.idp.defaults.authn_expiry": "5m", "xpack.idp.defaults.nameid_format": "urn:oasis:names:tc:SAML:2.0:nameid-format:transient", "xpack.idp.enabled": "false", "xpack.idp.entity_id": "", "xpack.idp.metadata.signing.keystore.alias": "", "xpack.idp.organization.display_name": "", "xpack.idp.organization.name": "", "xpack.idp.organization.url": "http:", "xpack.idp.privileges.application": "", "xpack.idp.privileges.cache.size": "100", "xpack.idp.privileges.cache.ttl": "90m", "xpack.idp.signing.keystore.alias": "", "xpack.idp.slo_endpoint.post": "https:", "xpack.idp.slo_endpoint.redirect": "https:", "xpack.idp.sp.cache.size": "1000", "xpack.idp.sp.cache.ttl": "60m", "xpack.idp.sp.wildcard.path": "wildcard_services.json", "xpack.idp.sso_endpoint.post": "https:", "xpack.idp.sso_endpoint.redirect": "https:", "xpack.inference.utility_thread_pool.core": "0", "xpack.inference.utility_thread_pool.keep_alive": "10m", "xpack.inference.utility_thread_pool.max": "1", "xpack.license.self_generated.type": "basic", "xpack.license.upload.types": [ "standard", "gold", "platinum", "enterprise", "trial" ], "xpack.ml.allocated_processors_scale": "1", "xpack.ml.autodetect_process": "true", "xpack.ml.datafeed_thread_pool.core": "1", "xpack.ml.datafeed_thread_pool.keep_alive": "1m", "xpack.ml.datafeed_thread_pool.max": "512", "xpack.ml.delayed_data_check_freq": "15m", "xpack.ml.enable_config_migration": "true", "xpack.ml.enabled": "true", "xpack.ml.inference_model.cache_size": "40%", "xpack.ml.inference_model.time_to_live": "5m", "xpack.ml.job_comms_thread_pool.core": "4", "xpack.ml.job_comms_thread_pool.keep_alive": "1m", "xpack.ml.job_comms_thread_pool.max": "2048", "xpack.ml.max_anomaly_records": "500", "xpack.ml.max_inference_processors": "50", "xpack.ml.max_lazy_ml_nodes": "0", "xpack.ml.max_machine_memory_percent": "30", "xpack.ml.max_ml_node_size": "0b", "xpack.ml.max_model_memory_limit": "0b", "xpack.ml.max_open_jobs": "512", "xpack.ml.min_disk_space_off_heap": "5gb", "xpack.ml.model_repository": "https://ml-models.elastic.co", "xpack.ml.native_inference_comms_thread_pool.core": "3", "xpack.ml.native_inference_comms_thread_pool.keep_alive": "1m", "xpack.ml.native_inference_comms_thread_pool.max": "312", "xpack.ml.nightly_maintenance_requests_per_second": "-1.0", "xpack.ml.node_concurrent_job_allocations": "2", "xpack.ml.persist_results_max_retries": "20", "xpack.ml.process_connect_timeout": "10s", "xpack.ml.use_auto_machine_memory_percent": "false", "xpack.ml.utility_thread_pool.core": "1", "xpack.ml.utility_thread_pool.keep_alive": "10m", "xpack.ml.utility_thread_pool.max": "2048", "xpack.monitoring.collection.ccr.stats.timeout": "10s", "xpack.monitoring.collection.cluster.stats.timeout": "10s", "xpack.monitoring.collection.enrich.stats.timeout": "10s", "xpack.monitoring.collection.index.recovery.active_only": "false", "xpack.monitoring.collection.index.recovery.timeout": "10s", "xpack.monitoring.collection.index.stats.timeout": "10s", "xpack.monitoring.collection.indices": [], "xpack.monitoring.collection.interval": "10s", "xpack.monitoring.collection.ml.job.stats.timeout": "10s", "xpack.monitoring.collection.node.stats.timeout": "10s", "xpack.monitoring.elasticsearch.collection.enabled": "true", "xpack.monitoring.history.duration": "168h", "xpack.monitoring.migration.decommission_alerts": "false", "xpack.monitoring.templates.enabled": "true", "xpack.notification.email.account.domain_allowlist": [ "*" ], "xpack.notification.email.default_account": "", "xpack.notification.email.html.sanitization.allow": [ "body", "head", "_tables", "_links", "_blocks", "_formatting", "img:embedded" ], "xpack.notification.email.html.sanitization.disallow": [], "xpack.notification.email.html.sanitization.enabled": "true", "xpack.notification.jira.default_account": "", "xpack.notification.pagerduty.default_account": "", "xpack.notification.reporting.interval": "15s", "xpack.notification.reporting.retries": "40", "xpack.notification.reporting.warning.enabled": "true", "xpack.notification.slack.default_account": "", "xpack.notification.webhook.additional_token_enabled": "false", "xpack.profiling.check_outdated_indices": "true", "xpack.profiling.enabled": "true", "xpack.profiling.query.details.max_slices": "16", "xpack.profiling.query.realtime": "true", "xpack.profiling.query.stacktrace.max_slices": "16", "xpack.profiling.templates.enabled": "false", "xpack.rollup.task_thread_pool.queue_size": "-1", "xpack.rollup.task_thread_pool.size": "1", "xpack.searchable.snapshot.allocate_on_rolling_restart": "false", "xpack.searchable.snapshot.cache.range_size": "32mb", "xpack.searchable.snapshot.cache.recovery_range_size": "128kb", "xpack.searchable.snapshot.cache.sync.interval": "60s", "xpack.searchable.snapshot.cache.sync.max_files": "10000", "xpack.searchable.snapshot.cache.sync.shutdown_timeout": "10s", "xpack.searchable.snapshot.shared_cache.count_reads": "true", "xpack.searchable.snapshot.shared_cache.decay.interval": "60s", "xpack.searchable.snapshot.shared_cache.max_freq": "100", "xpack.searchable.snapshot.shared_cache.min_time_delta": "60s", "xpack.searchable.snapshot.shared_cache.mmap": "false", "xpack.searchable.snapshot.shared_cache.range_size": "16mb", "xpack.searchable.snapshot.shared_cache.recovery_range_size": "128kb", "xpack.searchable.snapshot.shared_cache.region_size": "16mb", "xpack.searchable.snapshot.shared_cache.size": "0", "xpack.searchable.snapshot.shared_cache.size.max_headroom": "-1", "xpack.searchable_snapshots.cache_fetch_async_thread_pool.core": "0", "xpack.searchable_snapshots.cache_fetch_async_thread_pool.keep_alive": "30s", "xpack.searchable_snapshots.cache_fetch_async_thread_pool.max": "12", "xpack.searchable_snapshots.cache_prewarming_thread_pool.core": "0", "xpack.searchable_snapshots.cache_prewarming_thread_pool.keep_alive": "30s", "xpack.searchable_snapshots.cache_prewarming_thread_pool.max": "16", "xpack.security.audit.enabled": "false", "xpack.security.audit.logfile.emit_cluster_name": "false", "xpack.security.audit.logfile.emit_cluster_uuid": "true", "xpack.security.audit.logfile.emit_node_host_address": "false", "xpack.security.audit.logfile.emit_node_host_name": "false", "xpack.security.audit.logfile.emit_node_id": "true", "xpack.security.audit.logfile.emit_node_name": "false", "xpack.security.audit.logfile.events.emit_request_body": "false", "xpack.security.audit.logfile.events.exclude": [], "xpack.security.audit.logfile.events.include": [ "ACCESS_DENIED", "ACCESS_GRANTED", "ANONYMOUS_ACCESS_DENIED", "AUTHENTICATION_FAILED", "CONNECTION_DENIED", "TAMPERED_REQUEST", "RUN_AS_DENIED", "RUN_AS_GRANTED", "SECURITY_CONFIG_CHANGE" ], "xpack.security.authc.anonymous.authz_exception": "true", "xpack.security.authc.anonymous.roles": [], "xpack.security.authc.anonymous.username": "_anonymous", "xpack.security.authc.api_key.cache.hash_algo": "ssha256", "xpack.security.authc.api_key.cache.max_keys": "25000", "xpack.security.authc.api_key.cache.ttl": "24h", "xpack.security.authc.api_key.delete.interval": "24h", "xpack.security.authc.api_key.delete.retention_period": "7d", "xpack.security.authc.api_key.delete.timeout": "-1", "xpack.security.authc.api_key.doc_cache.ttl": "5m", "xpack.security.authc.api_key.enabled": "true", "xpack.security.authc.api_key.hashing.algorithm": "PBKDF2", "xpack.security.authc.password_hashing.algorithm": "BCRYPT", "xpack.security.authc.reserved_realm.enabled": "true", "xpack.security.authc.run_as.enabled": "true", "xpack.security.authc.service_token.cache.hash_algo": "ssha256", "xpack.security.authc.service_token.cache.max_tokens": "100000", "xpack.security.authc.service_token.cache.ttl": "20m", "xpack.security.authc.success_cache.enabled": "true", "xpack.security.authc.success_cache.expire_after_access": "1h", "xpack.security.authc.success_cache.size": "10000", "xpack.security.authc.token.delete.interval": "30m", "xpack.security.authc.token.delete.timeout": "-1", "xpack.security.authc.token.enabled": "false", "xpack.security.authc.token.thread_pool.queue_size": "1000", "xpack.security.authc.token.thread_pool.size": "1", "xpack.security.authc.token.timeout": "20m", "xpack.security.authz.store.privileges.cache.max_size": "10000", "xpack.security.authz.store.privileges.cache.ttl": "24h", "xpack.security.authz.store.roles.cache.max_size": "10000", "xpack.security.authz.store.roles.field_permissions.cache.max_size_in_bytes": "104857600", "xpack.security.authz.store.roles.has_privileges.cache.max_size": "1000", "xpack.security.authz.store.roles.negative_lookup_cache.max_size": "10000", "xpack.security.authz.timer.indices.enabled": "false", "xpack.security.authz.timer.indices.threshold.debug": "20ms", "xpack.security.authz.timer.indices.threshold.info": "100ms", "xpack.security.authz.timer.indices.threshold.warn": "200ms", "xpack.security.autoconfiguration.enabled": "true", "xpack.security.automata.cache.enabled": "true", "xpack.security.automata.cache.size": "10000", "xpack.security.automata.cache.ttl": "48h", "xpack.security.automata.max_determinized_states": "100000", "xpack.security.crypto.thread_pool.queue_size": "1000", "xpack.security.crypto.thread_pool.size": "2", "xpack.security.dls.bitset.cache.size": "10%", "xpack.security.dls.bitset.cache.ttl": "2h", "xpack.security.dls_fls.enabled": "true", "xpack.security.enabled": "true", "xpack.security.encryption.algorithm": "AES/CTR/NoPadding", "xpack.security.encryption_key.algorithm": "AES", "xpack.security.encryption_key.length": "128", "xpack.security.enrollment.enabled": "false", "xpack.security.filter.always_allow_bound_address": "true", "xpack.security.fips_mode.enabled": "false", "xpack.security.http.filter.allow": [], "xpack.security.http.filter.deny": [], "xpack.security.http.filter.enabled": "true", "xpack.security.http.ssl.enabled": "true", "xpack.security.operator_privileges.enabled": "false", "xpack.security.remote_cluster.filter.allow": [], "xpack.security.remote_cluster.filter.deny": [], "xpack.security.remote_cluster_client.ssl.enabled": "true", "xpack.security.remote_cluster_server.ssl.enabled": "true", "xpack.security.ssl.diagnose.trust": "true", "xpack.security.transport.filter.allow": [], "xpack.security.transport.filter.deny": [], "xpack.security.transport.filter.enabled": "true", "xpack.security.transport.ssl.enabled": "true", "xpack.security.user": null, "xpack.transform.num_transform_failure_retries": "10", "xpack.transform.transform_scheduler_frequency": "1s", "xpack.watcher.actions.bulk.default_timeout": "", "xpack.watcher.actions.index.default_timeout": "", "xpack.watcher.bulk.actions": "1", "xpack.watcher.bulk.concurrent_requests": "0", "xpack.watcher.bulk.flush_interval": "1s", "xpack.watcher.bulk.size": "1mb", "xpack.watcher.enabled": "true", "xpack.watcher.encrypt_sensitive_data": "false", "xpack.watcher.execution.default_throttle_period": "5s", "xpack.watcher.execution.scroll.size": "0", "xpack.watcher.execution.scroll.timeout": "", "xpack.watcher.index.rest.direct_access": "", "xpack.watcher.input.search.default_timeout": "", "xpack.watcher.internal.ops.bulk.default_timeout": "", "xpack.watcher.internal.ops.index.default_timeout": "", "xpack.watcher.internal.ops.search.default_timeout": "", "xpack.watcher.stop.timeout": "30s", "xpack.watcher.thread_pool.queue_size": "1000", "xpack.watcher.thread_pool.size": "1", "xpack.watcher.transform.search.default_timeout": "", "xpack.watcher.trigger.schedule.ticker.tick_interval": "500ms", "xpack.watcher.use_ilm_index_management": "true", "xpack.watcher.watch.scroll.size": "0" } } } ``` I currently have this value set to: ```auto "cluster.routing.allocation.balance.disk_usage": "2.0E-11", ``` What value would you recommend setting? ---
**Author:** ![hendry.lim](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendry.lim/32/71328_2.png) [@hendry.lim](https://discuss.elastic.co/u/hendry.lim)\ **Post date:** [February 6, 2024, 6:34am UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/6 "2024-02-06T06:34:39Z")
You can use the `PUT _cluster/settings` API to set that setting value to `1.0`. ---
**Author:** ![San9](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@San9](https://discuss.elastic.co/u/San9)\ **Post date:** [February 6, 2024, 8:23am UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/7 "2024-02-06T08:23:14Z")
Do I understand correctly that you need to execute the request: ```auto PUT _cluster/settings { "persistent": { "cluster.routing.allocation.balance.disk_usage": 1.0 } } ``` ??? now GET \_cluster/settings ```auto { "persistent": { "xpack": { "monitoring": { "collection": { "enabled": "true" } } } }, "transient": {} } ``` thanks ---
**Author:** ![hendry.lim](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendry.lim/32/71328_2.png) [@hendry.lim](https://discuss.elastic.co/u/hendry.lim)\ **Post date:** [February 6, 2024, 8:45am UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/8 "2024-02-06T08:45:57Z")
Yup, you should start seeing shards being moved around. ---
**Author:** ![San9](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@San9](https://discuss.elastic.co/u/San9)\ **Post date:** [February 7, 2024, 7:32am UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/9 "2024-02-07T07:32:26Z")
thanks, Now the situation has improved a little. It looks like this: ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/6/561d46594a670b7c61c791a1030bad9ee83de97b.png) I understand that the situation will not improve? from 150 to 279 GB ---
**Author:** ![hendry.lim](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendry.lim/32/71328_2.png) [@hendry.lim](https://discuss.elastic.co/u/hendry.lim)\ **Post date:** [February 26, 2024, 9:04am UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/10 "2024-02-26T09:04:40Z")
It really depends how your shard size distribution is like. If you have unbalanced shard size, then that will be the best you can get. ---
**Author:** ![San9](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@San9](https://discuss.elastic.co/u/San9)\ **Post date:** [February 29, 2024, 10:40am UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/11 "2024-02-29T10:40:41Z")
Сan you describe in simple words on what principle shards and indexes are distributed between nodes? According to the size of the fragments or their number? Since I’m constantly looking, I don’t have enough space on one node, and the rest are filled more or less evenly. In templates I usually use 2 or 4 shards. ---
**Author:** ![San9](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@San9](https://discuss.elastic.co/u/San9)\ **Post date:** [March 12, 2024, 9:53am UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/12 "2024-03-12T09:53:23Z")
now I have problem cluster status "yellow": ```auto { "decider": "disk_threshold", "decision": "NO", "explanation": "the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low.max_headroom=200gb], having less than the minimum required [200gb] free space, actual free: [169.8gb], actual used: [88.4%]" } ``` and ```auto node shards disk.indices disk.used disk.avail disk.total disk.percent v-elk-ed02 172 1tb 1.2tb 168.9gb 1.4tb 88 v-elk-ed03 743 1006.1gb 1tb 398.8gb 1.4tb 72 v-elk-ed01 743 1006.8gb 1tb 398.2gb 1.4tb 73 v-elk-ed04 658 1000.5gb 1tb 404.6gb 1.4tb 72 UNASSIGNED 4 ``` How can I move shards by hand? or does it not make any sense? on all other nodes there is another 400 GB of free space, but on this 02 it is constantly on the verge ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [March 12, 2024, 10:00am UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/13 "2024-03-12T10:00:33Z")
How large are your largest indices? How many shards do these have? Do you have an excessive amount of these allocated to the `v-elk-ed02` node? ---
**Author:** ![San9](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@San9](https://discuss.elastic.co/u/San9)\ **Post date:** [March 12, 2024, 1:03pm UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/14 "2024-03-12T13:03:58Z")
hi, We mostly use 4 shards, less often 2 shards 1 replica. as an example: ```auto index shard prirep state docs store dataset ip node fs-winlog-min-2023.12.19 3 p STARTED 22766633 7.3gb 7.3gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.27 0 p STARTED 7879602 2.8gb 2.8gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.27 1 p STARTED 7880659 2.8gb 2.8gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-login-2023.09 0 p STARTED 21620538 8.6gb 8.6gb 10.16.5.207 v-elk-ed02 my77-ucsyslog-2024.02 0 p STARTED 23930884 4.4gb 4.4gb 10.16.5.207 v-elk-ed02 my77-ucsyslog-2024.02 1 r STARTED 23926273 4.4gb 4.4gb 10.16.5.207 v-elk-ed02 .enrich-service-itsm-scom_policy2-1702291401499 0 r STARTED 1818 232.2kb 232.2kb 10.16.5.207 v-elk-ed02 fs-winlog-min-2024.01.07 2 p STARTED 23146161 6.3gb 6.3gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2024.01.07 3 p STARTED 23153260 6.3gb 6.3gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12_part 0 p STARTED 142772625 43.9gb 43.9gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12_part 2 p STARTED 142805430 43.7gb 43.7gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12_part 3 p STARTED 142772189 43.6gb 43.6gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.02 0 p STARTED 9952876 4.3gb 4.3gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.16 0 p STARTED 15773208 4.8gb 4.8gb 10.16.5.207 v-elk-ed02 filebeat-erp-tgb-v-dvf-jkupn10-2024.02 1 p STARTED 46387432 14.9gb 14.9gb 10.16.5.207 v-elk-ed02 exchange-v-dfhn-ik03-2024.02 0 r STARTED 2870895 4.5gb 4.5gb 10.16.5.207 v-elk-ed02 exchange-v-dfhn-ik04-2024.02 1 p STARTED 2870897 4.4gb 4.4gb 10.16.5.207 v-elk-ed02 exchange-v-dfhn-ik06-2024.02 0 p STARTED 2850112 4.4gb 4.4gb 10.16.5.207 v-elk-ed02 exchange-v-dfhn-ik05-2024.02 1 p STARTED 2852018 4.4gb 4.4gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.06 0 p STARTED 17317916 6.7gb 6.7gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.06 1 p STARTED 17319428 6.7gb 6.7gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.06 2 p STARTED 17318077 6.7gb 6.7gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.06 3 p STARTED 17315193 6.7gb 6.7gb 10.16.5.207 v-elk-ed02 .enrich-service-itsm-scom_policy3-1710216001646 0 r UNASSIGNED exchange-v-dfhn-ik03-2024.03 0 r STARTED 963723 2.7gb 2.7gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.29 0 p STARTED 13759706 5.2gb 5.2gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.29 2 p STARTED 13762261 5.2gb 5.2gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.29 3 p STARTED 13760778 5.2gb 5.2gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.27 2 p STARTED 29828398 9.6gb 9.6gb 10.16.5.207 v-elk-ed02 scom-srv-performance-2024.03 0 p STARTED 6716075 1.3gb 1.3gb 10.16.5.207 v-elk-ed02 scom-srv-performance-2024.03 1 p STARTED 6713107 1.3gb 1.3gb 10.16.5.207 v-elk-ed02 .monitoring-logstash-7-2024.03.06 0 r STARTED 11099840 1.7gb 1.7gb 10.16.5.207 v-elk-ed02 exchange-v-dfhn-ik03-2024.02 1 p STARTED 3442589 5.7gb 5.7gb 10.16.5.207 v-elk-ed02 ngfw-block_corp-2024.03.12 0 p STARTED 3830418 2.1gb 2.1gb 10.16.5.207 v-elk-ed02 ngfw-block_corp-2024.03.12 1 p STARTED 3835641 2.1gb 2.1gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-user-workstation-2023.07 0 r STARTED 7768633 2.7gb 2.7gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-user-workstation-2023.07 1 r STARTED 7770987 2.7gb 2.7gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-user-workstation-2023.05 0 p STARTED 7713667 2.7gb 2.7gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-user-workstation-2023.05 1 p STARTED 7711291 2.7gb 2.7gb 10.16.5.207 v-elk-ed02 .enrich-user-itsm-to-worksrtation_policy-1710216424733 0 r UNASSIGNED fs-winlog-2024.01.26 1 p STARTED 11494708 5gb 5gb 10.16.5.207 v-elk-ed02 .enrich-service-itsm-scom-wan_policy-1702291396696 0 r STARTED 1778 732.6kb 732.6kb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.24 1 p STARTED 14640708 5.9gb 5.9gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.24 2 p STARTED 14642932 5.9gb 5.9gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.24 3 p STARTED 14635500 5.9gb 5.9gb 10.16.5.207 v-elk-ed02 my78-ngfw-logs-2024.03.12 0 p STARTED 15357191 9.6gb 9.6gb 10.16.5.207 v-elk-ed02 exchange-v-dfhn-ik03-2024.02 0 p STARTED 3116061 4.8gb 4.8gb 10.16.5.207 v-elk-ed02 exchange-v-tujn-mn03-2024.02 1 r STARTED 3114365 4.8gb 4.8gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-user-workstation-2023.03-new 0 r STARTED 6811009 2.1gb 2.1gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-user-workstation-2023.03-new 1 r STARTED 6809977 2.1gb 2.1gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.11 0 p STARTED 17928852 6.2gb 6.2gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.11 1 p STARTED 17921865 6.1gb 6.1gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.11 2 p STARTED 17923558 6.2gb 6.2gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-user-workstation-2024.02 0 p STARTED 1503284 562.3mb 562.3mb 10.16.5.207 v-elk-ed02 winlogbeat-srv-user-workstation-2024.02 1 p STARTED 1505029 560mb 560mb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.31 0 p STARTED 14274498 3.9gb 3.9gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.31 1 p STARTED 14279931 3.9gb 3.9gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.31 2 p STARTED 14281505 4gb 4gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.31 3 p STARTED 14281024 4gb 4gb 10.16.5.207 v-elk-ed02 exchange-v-exch-fe03-2024.03 0 r STARTED 1152602 1.9gb 1.9gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.20 1 p STARTED 17157685 5.9gb 5.9gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.25 0 p STARTED 11936311 3.5gb 3.5gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.25 1 p STARTED 11940034 3.5gb 3.5gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.25 2 p STARTED 11933966 3.5gb 3.5gb 10.16.5.207 v-elk-ed02 ftp-logs-2023.05 1 p STARTED 65886090 9.1gb 9.1gb 10.16.5.207 v-elk-ed02 ngfw-block_corp-2024.03.09 0 p STARTED 6200291 3.1gb 3.1gb 10.16.5.207 v-elk-ed02 ngfw-block_corp-2024.03.09 1 p STARTED 6201501 3.1gb 3.1gb 10.16.5.207 v-elk-ed02 ngfw-block_corp-2024.03.09 2 p STARTED 6199340 3.1gb 3.1gb 10.16.5.207 v-elk-ed02 exchange-v-exch-fe02-2024.02 0 r STARTED 3466776 5.5gb 5.5gb 10.16.5.207 v-elk-ed02 exchange-v-exch-fe02-2024.02 1 p STARTED 3467269 5.5gb 5.5gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-sdc-2024.03.11 0 p STARTED 22224591 21.6gb 21.6gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-sdc-2024.03.11 1 p STARTED 22221427 21.6gb 21.6gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-user-workstation-geo-2023.06 0 p STARTED 7782933 3gb 3gb 10.16.5.207 v-elk-ed02 .enrich-user-workstation_policy-1702291423716 0 r STARTED 14945 762.5kb 762.5kb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.29 0 p STARTED 20725852 6.9gb 6.9gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.29 3 p STARTED 20733859 6.9gb 6.9gb 10.16.5.207 v-elk-ed02 filebeat-hjj-aud-v-rnu-perpn55-2024.03 0 r STARTED 10097015 2.1gb 2.1gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2024.01.06 2 p STARTED 23722486 6.6gb 6.6gb 10.16.5.207 v-elk-ed02 .monitoring-logstash-7-2024.03.08 0 p STARTED 11123504 1.7gb 1.7gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.08 2 p STARTED 43537630 15.9gb 15.9gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.08 3 p STARTED 43543578 15.8gb 15.8gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.15 1 p STARTED 11132703 5.1gb 5.1gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.15 2 p STARTED 11138909 5.1gb 5.1gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.03.07 2 p STARTED 15314092 5.6gb 5.6gb 10.16.5.207 v-elk-ed02 ngfw-block-2024.02 0 p STARTED 3187955 1.2gb 1.2gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.14 0 p STARTED 6406676 2.2gb 2.2gb 10.16.5.207 v-elk-ed02 sftp-logs-2023.06 1 p STARTED 3044754 496.5mb 496.5mb 10.16.5.207 v-elk-ed02 fs-winlog-2024.03.06 3 p STARTED 17156355 6.1gb 6.1gb 10.16.5.207 v-elk-ed02 .enrich-network_policy-1710216153396 0 r UNASSIGNED fs-winlog-2024.02.10 1 p STARTED 14309526 3.5gb 3.5gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.31 0 p STARTED 16868425 6.9gb 6.9gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.31 1 p STARTED 16867192 6.9gb 6.9gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.31 3 p STARTED 16871728 6.9gb 6.9gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.12 3 p STARTED 23861914 7.6gb 7.6gb 10.16.5.207 v-elk-ed02 .monitoring-logstash-7-2024.03.09 0 p STARTED 11085298 1.7gb 1.7gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-login-2024.02 0 p STARTED 16354188 5.8gb 5.8gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-login-2024.02 1 p STARTED 16354433 5.8gb 5.8gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.20 0 p STARTED 7829633 2.8gb 2.8gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.20 2 p STARTED 7831263 2.8gb 2.8gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.01 0 p STARTED 13597493 6.2gb 6.2gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.01 1 p STARTED 13597698 6.2gb 6.2gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.01 2 p STARTED 13595497 6.2gb 6.2gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.01 3 p STARTED 13598897 6.2gb 6.2gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.14 0 p STARTED 18793673 6.6gb 6.6gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.14 1 p STARTED 18791955 6.6gb 6.6gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.14 2 p STARTED 18791916 6.6gb 6.6gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.14 3 p STARTED 18790975 6.5gb 6.5gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.03 0 p STARTED 8428053 2.9gb 2.9gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.03 1 p STARTED 8427834 2.9gb 2.9gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.17 3 p STARTED 13038265 5.4gb 5.4gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.29 0 p STARTED 12842982 5.7gb 5.7gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.29 1 p STARTED 12854429 5.7gb 5.7gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.29 2 p STARTED 12850419 5.7gb 5.7gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.29 3 p STARTED 12847273 5.7gb 5.7gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.03.08 0 p STARTED 15430595 5gb 5gb 10.16.5.207 v-elk-ed02 .monitoring-es-7-2024.03.12 0 p STARTED 3381449 1.6gb 1.6gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.17 0 p STARTED 20179180 5.6gb 5.6gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.17 2 p STARTED 20184385 5.6gb 5.6gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.17 3 p STARTED 20172512 5.7gb 5.7gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.26 1 p STARTED 21323908 7.2gb 7.2gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.16 0 p STARTED 11923338 5.6gb 5.6gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2024.01.05 2 p STARTED 21640863 7.6gb 7.6gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2024.01.05 3 p STARTED 21635062 7.6gb 7.6gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-user-workstation-2023.04-new 0 p STARTED 2622042 837.6mb 837.6mb 10.16.5.207 v-elk-ed02 winlogbeat-srv-user-workstation-2023.04-new 1 r STARTED 2621311 850.4mb 850.4mb 10.16.5.207 v-elk-ed02 winlogbeat-srv-login-2023.04 0 p STARTED 18172958 7.4gb 7.4gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.17 1 p STARTED 13260382 2.9gb 2.9gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.17 2 p STARTED 13259929 2.9gb 2.9gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.17 3 p STARTED 13256654 2.9gb 2.9gb 10.16.5.207 v-elk-ed02 ftp-logs-2024.01 0 p STARTED 66868410 9.5gb 9.5gb 10.16.5.207 v-elk-ed02 ftp-logs-2024.01 1 p STARTED 66890587 9.5gb 9.5gb 10.16.5.207 v-elk-ed02 pexip-syslog-2024.03 0 p STARTED 1715168 1004.6mb 1004.6mb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.12 0 p STARTED 16710458 5.6gb 5.6gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.03.03 0 p STARTED 14138850 4gb 4gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.06 0 p STARTED 17750837 6.5gb 6.5gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.06 1 p STARTED 17759663 6.6gb 6.6gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.06 2 p STARTED 17758895 6.6gb 6.6gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.13 2 p STARTED 16295318 5.9gb 5.9gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2024.01.04 1 p STARTED 23888628 8.4gb 8.4gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2024.01.04 2 p STARTED 23898922 8.4gb 8.4gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.05 0 p STARTED 12283757 5.4gb 5.4gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.05 3 p STARTED 12287337 5.4gb 5.4gb 10.16.5.207 v-elk-ed02 scom-srv-performance-2023.12 1 p STARTED 14110420 2.8gb 2.8gb 10.16.5.207 v-elk-ed02 scom-srv-performance-2023.07 0 p STARTED 19977606 3.9gb 3.9gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-sdc-2024.03.12 0 p STARTED 12125137 12.2gb 12.2gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-sdc-2024.03.12 1 p STARTED 12127455 11.6gb 11.6gb 10.16.5.207 v-elk-ed02 exchange-v-exch-fe05-2024.03 0 p STARTED 987113 2.7gb 2.7gb 10.16.5.207 v-elk-ed02 rdweb-info-2024 0 p STARTED 2721776 804.9mb 804.9mb 10.16.5.207 v-elk-ed02 rdweb-info-2024 1 r STARTED 2721814 791.9mb 791.9mb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.12.14 1 p STARTED 22667037 7.1gb 7.1gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-sdc-2024.03.09 0 p STARTED 13722879 13.6gb 13.6gb 10.16.5.207 v-elk-ed02 winlogbeat-srv-sdc-2024.03.09 1 p STARTED 13727072 13.6gb 13.6gb 10.16.5.207 v-elk-ed02 filebeat-uap-v-yap-fgh01-2024.02 0 p STARTED 30164448 9.6gb 9.6gb 10.16.5.207 v-elk-ed02 filebeat-uap-v-yap-fgt01-2024.02 1 p STARTED 30167771 9.6gb 9.6gb 10.16.5.207 v-elk-ed02 exchange-v-etuo-cc01-2024.02 0 p STARTED 1484113 2.4gb 2.4gb 10.16.5.207 v-elk-ed02 exchange-v-etuo-cc01-2024.02 1 p STARTED 1488051 2.3gb 2.3gb 10.16.5.207 v-elk-ed02 scom-srv-performance-2024.01 0 p STARTED 17557018 3.5gb 3.5gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.25 0 p STARTED 17810307 4.4gb 4.4gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.25 1 p STARTED 17806223 4.4gb 4.4gb 10.16.5.207 v-elk-ed02 fs-winlog-min-2023.10 1 p STARTED 25995392 9.8gb 9.8gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.23 2 p STARTED 11634968 5.4gb 5.4gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.19 1 p STARTED 9904107 4.5gb 4.5gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.19 3 p STARTED 9906810 4.5gb 4.5gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.18 0 p STARTED 13950622 3.4gb 3.4gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.02.18 1 p STARTED 13950683 3.4gb 3.4gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.03.02 1 p STARTED 14154167 4gb 4gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.03.02 2 p STARTED 14167623 4gb 4gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.03.02 3 p STARTED 14165002 4gb 4gb 10.16.5.207 v-elk-ed02 ftp-logs-2023.12 0 p STARTED 68847167 9.7gb 9.7gb 10.16.5.207 v-elk-ed02 ftp-logs-2023.11 0 p STARTED 47315162 6.9gb 6.9gb 10.16.5.207 v-elk-ed02 ftp-logs-2023.11 1 p STARTED 47300321 6.9gb 6.9gb 10.16.5.207 v-elk-ed02 fs-winlog-2024.01.30 2 p STARTED 12555915 5.7gb 5.7gb 10.16.5.207 v-elk-ed02 filebeat-nap-v-gjk-webput7-2024.03 0 p STARTED 4063326 904.1mb 904.1mb 10.16.5.207 v-elk-ed02 filebeat-nap-v-gjh-webput7-2024.03 1 p STARTED 4061369 893.9mb 893.9mb 10.16.5.207 v-elk-ed02 ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 9, 2024, 1:04pm UTC](https://discuss.elastic.co/t/problem-elastic-uneven-distribution-of-the-data-node/352409/15 "2024-04-09T13:04:42Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.