We have an ELK server that periodically runs out of space. In the spring I worked out a method using curator to delete indices that were more than 30 days old.
But I still don't know much about ELK - Because everyone is working from home nowadays, I can't walk over to the devs and ask. But I still need to solve this problem.
When I run my curator, it frees up the disk space by deleting indices - but many of the indices (and stored queries apparently) are still needed and need to be manually recreated. I figure I could script the creation of the needed files using bash and CLI commands, but that isn't ideal. Is there a way to use curator, or some other tool, to just reduce the index size (clean out data more than 30 days old)?
Here is what I have now:
[root@ELK elasticsearch]# cat /etc/curator/config.yml
- port: 9200*
- url_prefix: *
- use_ssl: False*
- client_cert: /etc/elasticsearch/config/certs/elk/elk.crt*
- client_key: /etc/elasticsearch/config/certs/elk/elk.key*
- ssl_no_validate: False*
- http_auth: *
- timeout: 30*
- master_only: False*
- loglevel: INFO*
- logfile: /var/log/curator/curator_log*
- logformat: default*
- blacklist: ['elasticsearch', 'urllib3']*
[root@ELK elasticsearch]# cat /etc/curator/action.yml
- action: delete_indices*
- description: >-*
Delete indices older than 30 days (based on creation date).*
- filtertype: age*
Any help would be very much appreciated!