Hi, in my Case, to test the API Workaround. I create a complete new test elk-stack. And after that i will do a elasticsearch restart to test it and saw the behavior in the Logs.
:~# systemctl stop elasticsearch.service
:~# systemctl start elasticsearch.service
and saw in the Log
:~# tail -f /var/log/elasticsearch/elasticsearch.log
[...]
[2021-08-17T14:48:30,422][INFO ][o.e.i.g.LocalDatabases ] [devubu2004] initialized default databases [[GeoLite2-Country.mmdb, GeoLite2-City.mmdb, GeoLite2-ASN.mmdb]], config databases [[]] and watching [/etc/elasticsearch/ingest-geoip] for changes
[2021-08-17T14:48:30,444][INFO ][o.e.i.g.DatabaseRegistry ] [devubu2004] initialized database registry, using geoip-databases directory [/tmp/elasticsearch-808219805137404262/geoip-databases/YRNSL6wtTTC5J6Ili4f7nw]
[...]
[2021-08-17T14:48:39,689][INFO ][o.e.c.s.ClusterSettings ] [devubu2004] updating [ingest.geoip.downloader.enabled] from [true] to [false]
[...]
[2021-08-17T14:48:44,419][INFO ][o.e.i.g.GeoIpDownloader ] [devubu2004] updating geoip databases
[2021-08-17T14:48:44,437][INFO ][o.e.i.g.GeoIpDownloader ] [devubu2004] fetching geoip databases overview from [https://geoip.elastic.co/v1/database?elastic_geoip_service_tos=agree]
[2021-08-17T14:48:46,911][INFO ][o.e.i.g.GeoIpDownloader ] [devubu2004] updating geoip database [GeoLite2-ASN.mmdb]
[2021-08-17T14:48:46,939][INFO ][o.e.c.r.a.AllocationService] [devubu2004] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.geoip_databases][0]]]).
[2021-08-17T14:48:52,085][WARN ][o.e.p.PersistentTasksClusterService] [devubu2004] trying to update state on non-existing task geoip-downloader
[2021-08-17T14:48:52,091][ERROR][o.e.i.g.GeoIpDownloader ] [devubu2004] error updating geoip database [GeoLite2-ASN.mmdb]
org.elasticsearch.ResourceNotFoundException: the task with id geoip-downloader and allocation id 2 doesn't exist
at org.elasticsearch.persistent.PersistentTasksClusterService$4.execute(PersistentTasksClusterService.java:236) ~[elasticsearch-7.14.0.jar:7.14.0]
at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:48) ~[elasticsearch-7.14.0.jar:7.14.0]
at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:691) ~[elasticsearch-7.14.0.jar:7.14.0]
at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:313) ~[elasticsearch-7.14.0.jar:7.14.0]
at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:208) ~[elasticsearch-7.14.0.jar:7.14.0]
at org.elasticsearch.cluster.service.MasterService.access$000(MasterService.java:62) ~[elasticsearch-7.14.0.jar:7.14.0]
at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:140) ~[elasticsearch-7.14.0.jar:7.14.0]
at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:139) ~[elasticsearch-7.14.0.jar:7.14.0]
at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:177) ~[elasticsearch-7.14.0.jar:7.14.0]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:673) ~[elasticsearch-7.14.0.jar:7.14.0]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:241) ~[elasticsearch-7.14.0.jar:7.14.0]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:204) ~[elasticsearch-7.14.0.jar:7.14.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
[... three times ERROR...]
Is/should that ERROR now the normal behavior?
And after that i want to delete the new created und mybe useless Index
green open .geoip_databases EDd8AnYfS8-8epV7Opua_g 1 0 84 0 81.3mb 81.3mb
:~$ curl -XDELETE 'http://localhost:9200/.geoip_databases'
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Indices [.geoip_databases] use and access is reserved for system operations"}],"type":"illegal_argument_exception","reason":"Indices [.geoip_databases] use and access is reserved for system operations"},"status":400}
But i can't do that. Also is this the normal behavior?