# Curator 5.8 filters **URL:** \ **Category:** Elasticsearch\ **Tags:** curator\ **Created:** [February 12, 2020, 1:00pm UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982 "2020-02-12T13:00:56Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![fgonzalez](https://avatars.discourse-cdn.com/v4/letter/f/71e660/32.png) [@fgonzalez](https://discuss.elastic.co/u/fgonzalez)\ **Post date:** [February 12, 2020, 1:00pm UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/1 "2020-02-12T13:00:56Z")
Good afternoon, I am trying to configure truncation of the indexes with Curator 5.8 in an ELK 7.6 with the following configuration and there is no way, do you see any errors in the filters? Thank you!! ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/6/a/6a965cbfdc2d35484ee74eca45e0930590f271d5.png) ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/e/a/eafbc15af1372acf5e72d8e025befe58b8290eb0.png) ``` - filtertype: pattern kind: prefix value: logstash- - filtertype: age source: name direction: older timestring: '%Y.%m.%d' unit: days unit_count: 0 - filtertype: count count: 1 pattern: '^(.*)-\d{6}$' reverse: true ``` ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 12, 2020, 9:23pm UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/2 "2020-02-12T21:23:13Z")
There's no errors shown here, so it's not easy to tell what you are asking for help with. If you are receiving an error, please share it so I can help see what's going on. Otherwise I'm left to guess. How many indices do you actually have? The kibana display is truncated enough to only show one. The count filter only shows a `count` of `1`. Also, please don't use parenthesis in your regex pattern, as that creates a capture group which might collide with the one the filter tries to create. So, change `value: '^(\logstash-).*'` to `value: '^logstash-.*$'` and change `pattern: '^(.*)-\d{6}$'` to `pattern: '^.*-\d{6}$'` ---
**Author:** ![fgonzalez](https://avatars.discourse-cdn.com/v4/letter/f/71e660/32.png) [@fgonzalez](https://discuss.elastic.co/u/fgonzalez)\ **Post date:** [February 13, 2020, 7:47am UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/3 "2020-02-13T07:47:52Z")
> [@fgonzalez](#): > > '^(.\*)-\d{6}$' What I am looking for is simply to be able to truncate the logs that I cited above logstash, with the configuration that I had simply put me that I did not find any index (that's why I asked if the filter was wrong in relation to the name of the index that I put in the capture) Thanks for the help!! ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/9/c/9c0c8a79c17f8c841858026cd323b9c374d3e045.png) ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/0/b/0bf78b8943640bac407c191af845e614806e2b03.png) ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 13, 2020, 8:14am UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/4 "2020-02-13T08:14:57Z")
My mistake. You _do_ need a capture pattern in the `count` filter, but not in the `pattern` filter. I still don't know how many indices you're actually trying to purge. Curator doesn't "truncate logs." It sends API calls to delete indices. Nothing you've shown so far shows any of the matched indices from any of the filters. Perhaps turn on DEBUG logging? ---
**Author:** ![fgonzalez](https://avatars.discourse-cdn.com/v4/letter/f/71e660/32.png) [@fgonzalez](https://discuss.elastic.co/u/fgonzalez)\ **Post date:** [February 13, 2020, 9:41am UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/5 "2020-02-13T09:41:29Z")
> [@theuntergeek](#): > > My mistake. You _do_ need a capture pattern in the `count` filter, but not in the `pattern` filter. > > I still don't know how many indices you're actually trying to purge. Curator doesn't "truncate logs." It sends API calls to delete indices. Nothing you've shown so far shows any of the matched indices from any of the filters. Perhaps turn on DEBUG logging? ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/0/7/07cbe76df9ba8ae2be78d6dd4be1566194d58215.png) ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 13, 2020, 10:50am UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/6 "2020-02-13T10:50:52Z")
I don’t see the problem. You have kept one index, exactly as specified by your configuration. The log from Curator is telling you there is nothing to delete because there are no _other_ indices matching the pattern. ---
**Author:** ![fgonzalez](https://avatars.discourse-cdn.com/v4/letter/f/71e660/32.png) [@fgonzalez](https://discuss.elastic.co/u/fgonzalez)\ **Post date:** [February 13, 2020, 11:27am UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/7 "2020-02-13T11:27:15Z")
> [@theuntergeek](#): > > I don’t see the problem. You have kept one index, exactly as specified by your configuration. The log from Curator is telling you there is nothing to delete because there are no _other_ indices matching the pattern. But what I want is to erase that index (more ahead they will be more setting up the environment) with that pattern and not caught by curator ... ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 14, 2020, 1:08am UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/8 "2020-02-14T01:08:06Z")
Generally speaking, Curator users don't want to delete every index—they want to delete indices older than _n_ days, or similar. If you want to delete all Logstash indices, remove the other filters, except for the pattern filter. Also, the `prefix` filter automatically prepends `^` and appends `.*$` to whatever you put as `value`. So, it's not helpful to use `'^logstash-.*$'`. It should read: ```auto value: logstash- ``` And that's all. Omit the other filters if you want to delete the remaining Logstash index. With only a single index, you can easily delete it from within Kibana, too. ---
**Author:** ![fgonzalez](https://avatars.discourse-cdn.com/v4/letter/f/71e660/32.png) [@fgonzalez](https://discuss.elastic.co/u/fgonzalez)\ **Post date:** [February 14, 2020, 7:38am UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/9 "2020-02-14T07:38:19Z")
> [@theuntergeek](#): > > logstash- Good morning @theuntergeek, First of all thank you for the help you are giving me, it is curious and that's why I posted the question because even if I just leave the filter with logstash value- it doesn't eliminate the index, in fact if I put logstash-2020.02.10-000001 value (more exact impossible) does not eliminate it, it is as if it passed from that index. At the moment I only have that index but the idea is to integrate the ELK with a Suricata 5.0.1 and generate many logs with which I will be cleaning every 2 days or so ... Any idea why it happens to me if I just put it as a logstash pattern- Thank you very much, greetings! ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/e/1/e14a9257ef07eddcb1cf2446ea63d9f1b3cd26a6.png) ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 17, 2020, 3:21am UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/10 "2020-02-17T03:21:23Z")
Without DEBUG logging enabled, I won't be able to see what indices existed before/after filtering. You'll also have to attach a snippet or a file or something, as screenshots are not ideal for troubleshooting. ---
**Author:** ![fgonzalez](https://avatars.discourse-cdn.com/v4/letter/f/71e660/32.png) [@fgonzalez](https://discuss.elastic.co/u/fgonzalez)\ **Post date:** [February 17, 2020, 11:51am UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/11 "2020-02-17T11:51:57Z")
Good morning @theuntergeek, I leave you a pastebin of the two configuration files and a Curator debug with these, it is simply as if I did not recognize the filter and it happens, again thanks for your interest and help. Greetings! [https://pastebin.com/Mq8VWHpb](https://pastebin.com/Mq8VWHpb) [https://pastebin.com/Fce5Mqrv](https://pastebin.com/Fce5Mqrv) ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/7/9/79972c7b3181dfeb13971bd2f9fc50c6ebf2187b.png) ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 17, 2020, 12:07pm UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/12 "2020-02-17T12:07:15Z")
Okay, so this not DEBUG level. You can see from the Fce5Mqrv pastebin on line 18 that you have it set to `loglevel: INFO`. Change this to `loglevel: DEBUG` and the resulting output will have DEBUG information. ---
**Author:** ![fgonzalez](https://avatars.discourse-cdn.com/v4/letter/f/71e660/32.png) [@fgonzalez](https://discuss.elastic.co/u/fgonzalez)\ **Post date:** [February 17, 2020, 12:20pm UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/13 "2020-02-17T12:20:17Z")
[https://pastebin.com/ZAANzkqv](https://pastebin.com/ZAANzkqv) ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/d/3/d30f34cd0cd445249942d90a755cd2e2487e6ec4.png) I feel the awkwardness, I am relatively new to this. Greetings! ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 17, 2020, 12:41pm UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/14 "2020-02-17T12:41:23Z")
There it is. You can see that Curator is automatically filtering out indices with ILM policies. It appears that your index is being created and is already associated with an Index Lifecycle Management (ILM) policy. If so, you can possibly use ILM to manage your index lifecycle. If you want to continue to use Curator, you can [bypass this safety check](https://www.elastic.co/guide/en/elasticsearch/client/curator/5.8/ilm-and-curator.html) by setting `allow_ilm_indices: true` in the `options` for that action. ---
**Author:** ![fgonzalez](https://avatars.discourse-cdn.com/v4/letter/f/71e660/32.png) [@fgonzalez](https://discuss.elastic.co/u/fgonzalez)\ **Post date:** [February 17, 2020, 1:24pm UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/15 "2020-02-17T13:24:25Z")
> [@theuntergeek](#): > > `allow_ilm_indices: true` Now if @theuntergeek thank you very much for your help I didn't smell it taken out by myself. Greetings! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 16, 2020, 1:24pm UTC](https://discuss.elastic.co/t/curator-5-8-filters/218982/16 "2020-03-16T13:24:30Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.