Forgive me. Used ~v2 of the ELK and jumping forward and believe it's something simple but it's not jumping out at me after staring at the screen for a couple of days now. Get the same ~error from curator at the command line as what's recorded in debug mode in the log file and the action to close is not performed. Can someone please take a quick pass and advise what I'm doing wrong? Using v5.5.1 of curator and v6.2.2 of ES.
Response from debug and same general error from the command-line response. The rest of the debug looks "good" as the indexes "remain in the list", etc. and no errors seen until the very end. Can provide rest of debug log if requested.
2018-03-27 09:35:03,401 ERROR curator.cli run:184 Failed to complete action: close. <class 'curator.exceptions.FailedExecution'>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, 'cluster_block_exception', 'blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];')
Close indices older than 2 days (based on index name), for logstash-
- filtertype: pattern
- filtertype: age
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open logstash-2018.03.22 5kZGtUBPSH-AlUCQO0Xg3w 5 1 1376736 0 1gb 1gb
yellow open logstash-2018.03.12 FEev1P9ySA6XqUvjFY65Ww 5 1 5909374 0 3.9gb 3.9gb
yellow open logstash-2018.03.11 sRJBAMlmTvyEpxshTWLEtA 5 1 3258232 0 2.2gb 2.2gb
yellow open logstash-2018.03.08 XTO86zklToiYiisPIRpX6w 5 1 305 0 617.7kb 617.7kb
green open .monitoring-alerts-6 nTLlITQdTL6P930AQHB1Lw 1 0 2 1 12.9kb 12.9kb
green open .triggered_watches DRW4zTWERtqH6SyJENuIDQ 1 0 0 0 5.1kb 5.1kb
green open .watches -VP137M7ToiwJ7TCua8FcQ 1 0 6 0 24.5kb 24.5kb
close .watcher-history-7-2018.03.22 C0alcRK9QtGJCbzLKIjWoQ
yellow open logstash-2018.03.23 kKCz3ve1SFeLeTxtt4O-Wg 5 1 909987 0 706.9mb 706.9mb
yellow open logstash-2018.03.09 quU6dgvES5mqup828g1NRw 5 1 579 0 977.9kb 977.9kb
green open .monitoring-es-6-2018.03.22 bM8BUjstTiOUPXzIR4WOSQ 1 0 925 0 479.5kb 479.5kb
yellow open logstash-2018.03.26 L-xEeawmQoueFuktZTZmZg 5 1 4138979 0 3gb 3gb
yellow open logstash-2018.03.15 QadOJOHhRPyRTSsXjcQObQ 5 1 1811456 0 1.3gb 1.3gb
yellow open logstash-2018.03.27 660-4JaHS4eSo5R6bIGryQ 5 1 1876957 0 1.2gb 1.2gb
green open .kibana m9Lh3v-MSSKCFnlqKe2iig 1 0 11 5 71.6kb 71.6kb
yellow open logstash-2018.03.13 AcHi8k9LTtangpOEi_D3sw 5 1 28971 0 25.9mb 25.9mb
yellow open logstash-2018.03.10 xbDIUicyQSuRgLyN4a1Wqw 5 1 322 0 484.9kb 484.9kb
Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.
Or use markdown style like:
There's a live preview panel for exactly this reasons.
Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
With that said,
The error you provide here suggests security permissions are not sufficient to delete one of the indices as the user you logged in as.
Please still edit your post as described to make it more readable.
The guide as mentioned isn't exactly clear, figured it out. Appreciate the quick glance/analysis and catch on the permissions. Using the below example here on this forum, there's not a peep about permissions, just like it's supposed to work. Will go off and investigate/figure out permissions. Thank you.
I can appreciate your frustration that the guide posted did not mention permissions. Not everyone uses X-Pack or ECE, so not all users would have to consider permissions. Let me know if there's more I can help you with, especially regarding Curator.
Hmmm. Are you using Elastic Cloud Enterprise? This topic is posted in that forum. If you were using ECE, X-Pack would be a given, but you're talking about adding/removing X-Pack, so I'm a bit confused.
If you've uninstalled X-Pack on all nodes, kibana, logstash, etc, and still seeing this, did you restart the nodes after removing X-Pack? You probably need to do a full cluster stop/start to get things fully cleaned is all. After that, delete the .security* index/indices, and make sure to remove or comment out the xpack configuration lines in elasticsearch.yml, and other related files.
Ugh. My apology. Saw ES in the list and swear it was the generic ES group, not ECE. No, we are not use ECE (yet) nor in this scenario. Move, delete, close or whatever you like. We're rebuilding from scratch.
BTW, besides removing the index/indices, which is our problem to begin with due to permission denied, all the other steps were performed on all the apps. Clean install resolved it. (Also a chance to clean up/correct some of the documentation made during it's initial install.)