Set Limit for Date Filter in Elastic cluster

Hi
I have a general question!
I have a elastic with a lot of docs in index!

I want to set threshold on datetime filter and not allow users to execute heavy queries on cluster.

for example i set datetime_max_filter = 7days then if user enter date filter gt 7 days, kibana or elastic raise exception "please select time filter lte 7 days!"

Is this possible? (version 7.10.1)

There's a couple of open issues for similar requests. You could add a comment on these;

There is a rather convoluted way you might be able to accomplish this now if you have daily indices. For example, if you had indices like logstash-2022.07.13 for each day, you could create an alias that includes the last 7 days and some kind of scheduled task that updates that alias every day. Think of it as a "rolling 7-day alias". Then you create an Index Pattern (now called a Data View in the latest versions) for that alias. You could still also have a logstash-* Index Pattern that only admins had access to.

Here's an example that creates an alias named kibana_sample_data for all 3 of the sample data sets;

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "kibana_sample_data_ecommerce",
        "alias": "kibana_sample_data",
        "is_write_index": true
      }
    },
    {
      "add": {
        "index": "kibana_sample_data_flights",
        "alias": "kibana_sample_data"
      }
    },
    {
      "add": {
        "index": "kibana_sample_data_logs",
        "alias": "kibana_sample_data"
      }
    }
  ]
}

The script to do the alias would require some date math and would depend on what language you're most comfortable with.

Regards,
Lee

1 Like

I got pretty close with this bash line;

for i in {0..6}; do echo curl -XPOST https://elastic:changeit@localhost:9200/_aliases -k -H \'"Content-Type: application/json"\' -d \'"{ \"actions\": [ { \"add\": { \"index\": \"logstash-$(date -v-${i}d '+%Y.%m.%d')-000001\", \"alias\": \"test\" } } ] }"\'; done

It outputs lines to the console like this;

curl -XPOST https://elastic:changeit@localhost:9200/_aliases -k -H 'Content-Type: application/json' -d '{ "actions": [ { "add": { "index": "logstash-2022.07.14-000001", "alias": "test" } } ] }'
curl -XPOST https://elastic:changeit@localhost:9200/_aliases -k -H 'Content-Type: application/json' -d '{ "actions": [ { "add": { "index": "logstash-2022.07.13-000001", "alias": "test" } } ] }'
curl -XPOST https://elastic:changeit@localhost:9200/_aliases -k -H 'Content-Type: application/json' -d '{ "actions": [ { "add": { "index": "logstash-2022.07.12-000001", "alias": "test" } } ] }'
curl -XPOST https://elastic:changeit@localhost:9200/_aliases -k -H 'Content-Type: application/json' -d '{ "actions": [ { "add": { "index": "logstash-2022.07.11-000001", "alias": "test" } } ] }'
curl -XPOST https://elastic:changeit@localhost:9200/_aliases -k -H 'Content-Type: application/json' -d '{ "actions": [ { "add": { "index": "logstash-2022.07.10-000001", "alias": "test" } } ] }'
curl -XPOST https://elastic:changeit@localhost:9200/_aliases -k -H 'Content-Type: application/json' -d '{ "actions": [ { "add": { "index": "logstash-2022.07.09-000001", "alias": "test" } } ] }'
curl -XPOST https://elastic:changeit@localhost:9200/_aliases -k -H 'Content-Type: application/json' -d '{ "actions": [ { "add": { "index": "logstash-2022.07.08-000001", "alias": "test" } } ] }'

And if I copy/paste those lines, they work and create the alias which allows me to query those 7 indices. But my attempts to remove the echo from that command and have it just work have failed with syntax errors. I'm sure it has something to do with escaping the quotes. Maybe a different language would be better.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.