Hi,
We have an index .ds-logs-network.azure-default-000001 that is returning:
illegal_state_exception: index, alias, and data stream names need to be unique, but the following duplicates were found [logs-network.azure-default (alias of [.ds-logs-network.azure-default-000002/N11Il5vtQ-GOVy-_fcXseA]) conflicts with data stream]
This index .ds-logs-network.azure-default-000002 doesn't exist yet.
We believe this is because index was initially created with an old template and then converted to data stream backing index (if that is even possible). We can see from the screenshot that it has alias from legacy index template: logs-network_azure_firewall-default
To my understanding .ds indexes shouldn't have aliases.
Legacy index template had an order of 0.
I've tried deleting that alias with API but it returns: "reason" : "The provided expressions [.ds-logs-network.azure-default-000001] match a backing index belonging to data stream [logs-network.azure-default]. Data streams and their backing indices don't support aliases."
Forcing rollover with API also doesn't work.
Cloning also won't work because:
The current write index on a data stream cannot be cloned. In order to clone the current write index, the data stream must first be rolled over so that a new write index is created and then the previous write index can be cloned.
Creating new logs-network.azure-default-000002 index with API didn't work.
Should we try to reindex it somehow into some new index with different name just to have it backed up, and then delete it?
What can we do to fix this?
Thanks,
Dane
Full stack trace:
java.lang.IllegalStateException: index, alias, and data stream names need to be unique, but the following duplicates were found [logs-network.azure-default (alias of [.ds-logs-network.azure-default-000002/N11Il5vtQ-GOVy-_fcXseA]) conflicts with data stream]
at org.elasticsearch.cluster.metadata.Metadata$Builder.build(Metadata.java:1464)
at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.clusterStateCreateIndex(MetadataCreateIndexService.java:886)
at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.lambda$applyCreateIndexWithTemporaryService$3(MetadataCreateIndexService.java:433)
at org.elasticsearch.indices.IndicesService.withTempIndexService(IndicesService.java:619)
at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexWithTemporaryService(MetadataCreateIndexService.java:407)
at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequestWithV2Template(MetadataCreateIndexService.java:527)
at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:357)
at org.elasticsearch.action.admin.indices.rollover.MetadataRolloverService.rolloverDataStream(MetadataRolloverService.java:160)
at org.elasticsearch.action.admin.indices.rollover.MetadataRolloverService.rolloverClusterState(MetadataRolloverService.java:100)
at org.elasticsearch.action.admin.indices.rollover.TransportRolloverAction$1$1.execute(TransportRolloverAction.java:138)
at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:47)
at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:702)
at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:324)
at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:219)
at org.elasticsearch.cluster.service.MasterService.access$000(MasterService.java:73)
at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:151)
at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150)
at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188)
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:684)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)