Failed to put mappings on indices metricbeat-6.6.1-2019.11.15

Elastic Stack Elasticsearch
1

I just performed a rolling update on our elastic cluster from 6.6.1 to 6.8 to 7.4 and now I am no longer creating new indices from metricbeat or auditbeat starting when I upgraded the beats clients from 6.8 to 7.4. The client side looks fine it is connecting to the server and reporting every 30 seconds as configured. And when I look at discover I see logs, what I am not seeing is new indices when looking at index management. I have one new index "metricbeat-7.4.2-2019.11.13-0000001" from the day I installed the updates, same for auditbeat and none since then.

I found the following error on the server side but in my googling I saw mostly things related to logstash. I am not using any custom index templates the index template was created by running metricbeat setup --index-management

[2019-11-15T17:47:26,206][DEBUG][o.e.a.a.i.m.p.TransportPutMappingAction] [elastic01] failed to put mappings on indices [[[metricbeat-6.6.1-2019.11.15/JjjvYxD8SpGFiva_cx3ULw]]], type [doc]
java.lang.IllegalArgumentException: Rejecting mapping update to [metricbeat-6.6.1-2019.11.15] as the final mapping would have more than 1 type: [_doc, doc]
	at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:272) ~[elasticsearch-7.4.2.jar:7.4.2]
	at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:238) ~[elasticsearch-7.4.2.jar:7.4.2]
	at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:702) ~[elasticsearch-7.4.2.jar:7.4.2]
	at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:324) ~[elasticsearch-7.4.2.jar:7.4.2]
	at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:219) [elasticsearch-7.4.2.jar:7.4.2]
	at org.elasticsearch.cluster.service.MasterService.access$000(MasterService.java:73) [elasticsearch-7.4.2.jar:7.4.2]
	at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:151) [elasticsearch-7.4.2.jar:7.4.2]
	at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-7.4.2.jar:7.4.2]
	at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-7.4.2.jar:7.4.2]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:703) [elasticsearch-7.4.2.jar:7.4.2]
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) [elasticsearch-7.4.2.jar:7.4.2]
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedR       unnable.run(PrioritizedEsThreadPoolExecutor.java:215) [elasticsearch-7.4.2.jar:7.4.2]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:830) [?:?]
2

So turns out a few things are going on,

  1. Was a change in Index Lifecycle Policies. As of 7.0 indices are not created on a daily basis. The default policy is every 50 GB or 30 days, which ever happens first.
  2. I still have some old 6.6.1 clients in the environment that I need to update.
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.