# Curator throws exception when trying to delete indices based on timestamp **URL:** \ **Category:** Elasticsearch\ **Created:** [December 27, 2017, 5:04pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343 "2017-12-27T17:04:08Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![ikarthikb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikarthikb/32/47696_2.png) [@ikarthikb](https://discuss.elastic.co/u/ikarthikb)\ **Post date:** [December 27, 2017, 5:04pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/1 "2017-12-27T17:04:09Z")
I have an actions yml file that tries to delete ES indices that are more than 5 minutes old. My ES docs contain a field called timestamp which is used to identify the records. But when curator runs, I get an exception > "Failed to complete action: delete\_indices. \: int() arguent must be a string or a number, not 'NoneType' Here is how my actions look like ``` actions: 1: action: delete_indices description: >- Delete indices older than 5 minutes. options: ignore_empty_list: True timeout_override: continue_if_exception: False disable_action: False filters: - filtertype: age source: field_stats field: '@timestamp' stats_result: min_value direction: older unit: minutes unit_count: 5 ``` Any idea what I might be doing wrong? Thanks Kay ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [December 27, 2017, 5:36pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/2 "2017-12-27T17:36:45Z")
Without seeing more context from the logs, I can't be sure. More information would be helpful. But seeing your filters, you are _only_ filtering by `age`, and that means that _all_ of your indices are being parsed for that `@timestamp` field. This includes indices like `.kibana` and `.security` and `.watch...`. I am guessing that the filter is not able to see an `@timestamp` field in indices like that, and that is resulting in your error. You should use a `pattern` filter before the `age` filter to ensure that only indices which contain that field are selected for performing a `field_stats` operation. No, [Curator doesn't automatically do this for you](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/fe_field.html): > The value of this setting must be a timestamp field name. This field must be present in the indices being filtered or an exception will be raised, and execution will halt. ---
**Author:** ![ikarthikb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikarthikb/32/47696_2.png) [@ikarthikb](https://discuss.elastic.co/u/ikarthikb)\ **Post date:** [December 27, 2017, 5:47pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/3 "2017-12-27T17:47:11Z")
Appreciate the quick response. Your reply makes sense. So i changed my actions to this ``` filters: -filtertype: pattern kind: prefix value: myindex exclude: - filtertype: age source: field_stats field: '@timestamp' stats_result: min_value direction: older unit: minutes unit_count: 5 ``` Now curator is unable to match any documents, which makes me question my understanding of filters. Can I interpret those filters as "Find all docs whose index name starts with 'myindex' and if its timestamp field is older than 5 minutes, delete it" A different question, do i have to say '@timestamp' in field or can i leave the @ part? Thanks K ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [December 27, 2017, 6:01pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/4 "2017-12-27T18:01:28Z")
Curator doesn't match documents. It matches indices. In the case of the `field_stats` action, it matches indices which have documents with fields having timestamp values meeting the specified criteria. It sounds like you were expecting Curator do delete _documents_ but that is not how it works. Curator doesn't delete documents—it deletes whole indices. As an explanation, the pattern filter you specified will look for indices which start with `myindex` (`prefix`). Do you have indices that start with that? e.g. `myindex-1`, `myindex-2017.12.26`, `myindex-something-here`? ---
**Author:** ![ikarthikb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikarthikb/32/47696_2.png) [@ikarthikb](https://discuss.elastic.co/u/ikarthikb)\ **Post date:** [December 27, 2017, 7:49pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/5 "2017-12-27T19:49:34Z")
Thanks for the clarification. Yes i do have myindex as the index prefix for all my documents. After i replaced field: '@timestamp' with field: timestamp, my indices are getting deleted as expected. WRT to deleting the actual documents (I understand that curator does not do this) what would be a good approach? In my situation, I get documents every five minutes all day, I would like to purge (delete) all the documents and indices within elasticsearch at the end of the day so that I can keep my memory usage under control. Thanks again ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [December 27, 2017, 8:48pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/6 "2017-12-27T20:48:50Z")
If you are having a constant stream of documents coming in, you should not be deleting documents, but rather use time-series indices — they could be hourly, for example. You would then use Curator to delete indices when they are no longer needed. ---
**Author:** ![ikarthikb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikarthikb/32/47696_2.png) [@ikarthikb](https://discuss.elastic.co/u/ikarthikb)\ **Post date:** [December 27, 2017, 9:11pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/7 "2017-12-27T21:11:15Z")
Currently, all my documents share the same index. And I use time timestamp field to match what I want to delete. Your suggestion would mean that all the documents under a given hour would have the same index and when I delete the index based on age, what happens to the documents under that index? do they get purged automatically? Thanks K ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [December 27, 2017, 11:43pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/8 "2017-12-27T23:43:38Z")
If an index gets deleted, all documents in the index go with it. It's semi-analogous to a SQL drop table vs. doing a SQL delete clause. Dropping a table (or index, as is the case for Elasticsearch) is always going to be more performant than doing a bunch of atomic deletes. ---
**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\ **Post date:** [December 28, 2017, 3:03am UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/9 "2017-12-28T03:03:01Z")
as a time . this output no indices find. ---
**Author:** ![Kim-Kruse-Hansen](https://avatars.discourse-cdn.com/v4/letter/k/f1d935/32.png) [@Kim-Kruse-Hansen](https://discuss.elastic.co/u/Kim-Kruse-Hansen)\ **Post date:** [January 22, 2018, 6:41am UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/10 "2018-01-22T06:41:53Z")
Hi I have also had this message running curator. The reason for me was that index was empty. Is that intended behaviour? I would like to recommend that it would be ignored in case of empty index in order not to stop the pipeline. Rest regards Kim ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [January 22, 2018, 7:12am UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/11 "2018-01-22T07:12:37Z")
> [@Kim-Kruse-Hansen](#): > > Is that intended behaviour? Yes it is, and it is [explicitly documented](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/fe_field.html): > This field must be present in the indices being filtered or an exception will be raised, and execution will halt. Why do you have empty indices around eating up valuable JVM resources? ---
**Author:** ![tarp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarp/32/51884_2.png) [@tarp](https://discuss.elastic.co/u/tarp)\ **Post date:** [January 23, 2018, 3:32pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/12 "2018-01-23T15:32:19Z")
Hi, I followed this thread because I have the same issue. For whatever reason, I may have an index where @timestamp does not exist. I think if curator doesn't find it, it should ignore that index. I'm transitioning my indexing to active indices using aliases. I have pre-setup the initial empty index and alias for the development teams. When they move there code way from index-yyyy-mm-dd to using the active alias they will write docs with @timestamp. But for a few days the index will be empty. Also, I may have a development team that writes their time with 'timestamp', Until I figure out they are doing this wrong, then the curator job will fail when it hits that index. One more question, in version 6, field\_stats is deprecated, what is the alternative for curator? thanks, Tim ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [January 23, 2018, 8:37pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/13 "2018-01-23T20:37:47Z")
> [@tarp](#): > > One more question, in version 6, field\_stats is deprecated, what is the alternative for curator? Curator is calling it the same thing, but it now uses a query to accomplish the same thing. > [@tarp](#): > > For whatever reason, I may have an index where @timestamp does not exist. I think if curator doesn't find it, it should ignore that index. It is non-trivial to add this functionality to Curator. But I'm getting a lot of requests for it. It _may_ make it into 6.0. You'll have to wait and see. ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [January 23, 2018, 8:39pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/14 "2018-01-23T20:39:33Z")
You all understand the danger inherent in sending indices that _won't_ match a query for a timestamp to a tool that will be deleting indices based on timestamps it finds? This is not a recommended practice, to have the tool ignore something that shouldn't be happening. ---
**Author:** ![tarp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarp/32/51884_2.png) [@tarp](https://discuss.elastic.co/u/tarp)\ **Post date:** [January 23, 2018, 8:48pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/15 "2018-01-23T20:48:15Z")
I'm not really understanding the danger part. I have some old ES clusters at AWS that I had to write my own delete in this way. Curator doesn't work with the early 5.x ES instances at AWS. Anyway, I would iterate through the list and query each index. If it passed my "check the @timestamp" , it got added to the delete list. If it got no results, that was the same as failing the check. In a perfect world ES could enforce this so we couldn't have a stray index throw off this job. But the way it is, ES is more than happy to let you not configure @timestamp or call it 'timestamp'. ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [January 23, 2018, 9:05pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/16 "2018-01-23T21:05:40Z")
> ES is more than happy to let you not configure @timestamp or call it 'timestamp' Well, yes. It's meant to be configurable. So is the `field` directive for Curator, so you can pick whatever timestamp field you've chosen. Consistency in naming is something the end user has to enforce, rather then Elasticsearch. ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [January 23, 2018, 9:09pm UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/17 "2018-01-23T21:09:56Z")
> [@tarp](#): > > iterate through the list and query each index And this is where things broke for Curator. In \<5.4 versions, it used the `field_stats` API, which allowed the queries to be in bulk form, rather than needing to iterate over each index. Since iteration is what Elasticsearch mandated as the replacement for `field_stats`, Curator switched to doing that in 5.4. Adding the code to remove them from the actionable list is easiest than for `field_stats`, but still not something I'm eager to dive into. I'm trying to write big changes for 6.0, not handle edge cases for 5.x still. ---
**Author:** ![Kim-Kruse-Hansen](https://avatars.discourse-cdn.com/v4/letter/k/f1d935/32.png) [@Kim-Kruse-Hansen](https://discuss.elastic.co/u/Kim-Kruse-Hansen)\ **Post date:** [January 24, 2018, 10:28am UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/18 "2018-01-24T10:28:32Z")
Did you consider the possibility that a rollover has just occurred seconds ago and no data has been pushed to the new index yet ? And then the pipeline is halted because of no data yet? There is no valuable JVM resources being eaten up. Regards Kim ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 21, 2018, 10:28am UTC](https://discuss.elastic.co/t/curator-throws-exception-when-trying-to-delete-indices-based-on-timestamp/113343/19 "2018-02-21T10:28:39Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.