# Issues when running curator 8.0.17 **URL:** \ **Category:** Elastic Search\ **Tags:** curator\ **Created:** [January 9, 2025, 12:42pm UTC](https://discuss.elastic.co/t/issues-when-running-curator-8-0-17/372996 "2025-01-09T12:42:39Z")\ **Posts on this page:** 1\ **Showing post:** 15
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 5, 2025, 6:45pm UTC](https://discuss.elastic.co/t/issues-when-running-curator-8-0-17/372996/15 "2025-02-05T18:45:14Z")
I really do not know why the `value: "^\\.*"` pattern isn't working for you: ```yaml actions: 1: description: "This action should not affect anything. It's here to show filter behavior" action: open options: allow_ilm_indices: True filters: - filtertype: pattern kind: regex exclude: true value: "^\\.*" ``` Sample of debug log output from my own cluster running 8.17.0: ```auto 2025-02-05 11:04:42,769 DEBUG curator.indexlist iterate_filters:1452 Iterating over a list of filters 2025-02-05 11:04:42,769 DEBUG curator.indexlist iterate_filters:1457 All filters: [{'filtertype': 'pattern', 'kind': 'regex', 'exclude': True, 'value': '^\\.*'}] 2025-02-05 11:04:42,773 DEBUG curator.indexlist filter_by_regex:620 Filtering indices by regex 2025-02-05 11:04:42,773 DEBUG curator.indexlist empty_list_check:448 Checking for empty list 2025-02-05 11:04:42,773 DEBUG curator.indexlist working_list:459 Generating working list of indices 2025-02-05 11:04:42,773 DEBUG curator.indexlist filter_by_regex:637 Filter by regex: Index: .ds-metrics-kubernetes.node-default-2024.11.24-000002 2025-02-05 11:04:42,773 DEBUG curator.indexlist __not_actionable:58 Index .ds-metrics-kubernetes.node-default-2024.11.24-000002 is not actionable, removing from list. 2025-02-05 11:04:42,773 DEBUG curator.indexlist filter_by_regex:637 Filter by regex: Index: .ds-metrics-kubernetes.state_daemonset-k8s-2025.02.04-000004 2025-02-05 11:04:42,773 DEBUG curator.indexlist __not_actionable:58 Index .ds-metrics-kubernetes.state_daemonset-k8s-2025.02.04-000004 is not actionable, removing from list. ... ``` There's a lot more of the same that follows. As configured, this is filtering indices that start with a `.`. Let's go back to my earlier recommendation to use a `search_pattern`. Since we're having some issues with the regex pattern not filtering indices that start with a `.` we will take the other recommended route. > [@theuntergeek](#): > > **Use a `search_pattern` (now called [multi-target syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#api-multi-index))** > You can use an Elasticsearch [multi-target syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#api-multi-index) to include/exclude indices this way: > > ```auto > --- > actions: > 1: > action: delete_indices > description: "Example of search_pattern" > options: > search_pattern: '*,-.geoip_databases,-.fleet*,-.kibana*' > > ``` > > This search pattern begins by including all (`*`) indices, but then explicitly excludes (`-`) `.geoip_databases`, anything starting with `.fleet`, and anything starting with `.kibana`. You must start with _some_ indices before you can exclude. In this example, the wildcard for all is used first, and then the exclusions are listed. Initially, you shared a warning level log message showing system indices that should not be accessed directly: ```auto /usr/local/lib/python3.8/dist-packages/curator/indexlist.py:149: ElasticsearchWarning: this request accesses system indices: [.apm-agent-configuration, .async-search, .kibana_7.12.0_001, .kibana_7.17.16_001, .reporting-2023-02-12, .reporting-2024-01-07, .triggered_watches, .watches], but in a future major version, direct access to system indices will be prevented by default return self.client.indices.get_settings(index=to_csv(data)) ``` This is only a warning, so it would not halt execution. But we should be in the habit of heeding these warnings and not directly accessing the named indices. Additionally, the `.geoip_databases` index and any other system index should also be omitted from consideration, including `.fleet*`, `.kibana*`, `.apm-agent-configuration`, `.async-search`, `.reporting-*`, `.triggered_watches`, and `.watches`. As such, I have taken the liberty of creating a `search_pattern` that includes all indices _except_ the names and patterns preceded by a `-` (negating the pattern/name): ```yaml search_pattern: '*,-.geoip_databases,-.fleet*,-.kibana*,-.apm-agent-configuration,-.async-search,-.reporting-*,-.triggered_watches,-.watches' ``` Please make your `actions.yml` file look like this: ```yaml --- actions: 1: action: delete_indices description: "Delete indices older than 31 days" options: allow_ilm_indices: true ignore_empty_list: true disable_action: false search_pattern: '*,-.geoip_databases,-.fleet*,-.kibana*,-.apm-agent-configuration,-.async-search,-.reporting-*,-.triggered_watches,-.watches' filters: - filtertype: age source: name direction: older timestring: '%Y.%m.%d' unit: days unit_count: 31 ``` I have tested this with the `--dry-run` flag in my own cluster and it does match indices by `age` with this pattern as expected. ## Using `search_pattern` effectively Running a single action is simple and easy, and if you don't have too many indices it runs rather quickly. But keep in mind that with an `age` filter, Curator grabs metadata for all indices found by `search_pattern` before proceeding to the filters. On my own cluster, 529 indices were found by this `search_pattern`, and it took over 19 seconds to run, generating 2065 log lines in DEBUG mode. Curator did the work to gather the metadata on all 529 indices before filtering by age. But if I only needed to operate on a subset of all indices? Or what if I wanted my logs to be much shorter? By updating `search_pattern` to only gather indices starting with `.ds-metrics-kubernetes*`, I can simplify the `search_pattern`, shorten execution time, and reduce the log output size: ```yaml --- actions: 1: action: delete_indices description: "Delete indices older than 31 days" options: allow_ilm_indices: true ignore_empty_list: true disable_action: false search_pattern: '.ds-metrics-kubernetes*' filters: - filtertype: age source: name direction: older timestring: '%Y.%m.%d' unit: days unit_count: 31 ``` In my cluster, this shortened the list of indices needing metadata to 60, and execution time dropped to 2.6 seconds, generating only 284 log lines in DEBUG mode. You could just as easily expand the `search_pattern` to include more indices you know you will be working on: ```yaml search_pattern: '.ds-metrics-kubernetes*,.ds-metrics-system*' ``` The index count climbed to 82, execution time was 3.6s, and log line count was 371. Please use this feature to your advantage! --- _[View the full topic](https://discuss.elastic.co/t/issues-when-running-curator-8-0-17/372996)._