hunsw
September 30, 2021, 11:04am
1
After upgrading Elasticsearch from 7.10 to 7.14, Curator 5.4.8 has stopped working: simple delete jobs, create alias jobs, etc. that previously worked fine are now throwing this error:
2021-09-29 15:12:25,343 DEBUG curator.indexlist iterate_over_stats:124 Index: redacted-x-2021.09.13 Size: xxxx Docs: xx
2021-09-29 15:12:25,343 DEBUG curator.indexlist iterate_over_stats:124 Index: redacted-other-2021.07.23 Size: xxxx Docs: xx
2021-09-29 15:12:26,134 ERROR curator.cli run:213 Failed to complete action: delete_indices. <class 'KeyError'>: 'indices'
Failed to complete action: delete_indices. <class 'KeyError' >: 'indices'
The jobs have 2 filters: regex pattern and age (older+) and it had been working for a year or two.
hunsw
September 30, 2021, 11:06am
2
It seems that the last line before the error is (around) here:
"""
Populate `index_info` with index `size_in_bytes`, `primary_size_in_bytes` and doc count
information for each index.
"""
self.loggit.debug('Getting index stats')
self.empty_list_check()
# Subroutine to do the dirty work
def iterate_over_stats(stats):
for index in stats['indices']:
size = stats['indices'][index]['total']['store']['size_in_bytes']
docs = stats['indices'][index]['total']['docs']['count']
primary_size = stats['indices'][index]['primaries']['store']['size_in_bytes']
self.loggit.debug(
'Index: {0} Size: {1} Docs: {2} PrimarySize: {3}'.format(
index, utils.byte_size(size), docs, utils.byte_size(primary_size)
)
)
self.index_info[index]['size_in_bytes'] = size
self.index_info[index]['docs'] = docs
self.index_info[index]['primary_size_in_bytes'] = primary_size
hunsw
September 30, 2021, 11:38am
3
I had to download the source and wrap the part in question in a try-catch (except) block and it's working this way.
I'm doing further investigation and leaving this topic open until I fully understand what was the root cause.
theuntergeek
September 30, 2021, 3:43pm
4
How were you installing Curator before downloading source to test this?
hunsw
October 1, 2021, 8:53am
5
The error is there with a freshly git cloned Curator too. It seems that the action is irrelevant since it dies sooner.
In indexlist.py, around L125 the stats var is an empty dictionary. Two times out of the 1906 indices I have. def iterate_over_stats(stats):
Edit: I'm not a black belt Python developer, so I'm not sure where 'stats' come from in that subroutine.
This sounds like 2 of your indices do not have the requisite data, or access to that data is possibly blocked by policy or permission.
What two indices are they?
system
October 29, 2021, 2:02pm
7
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
