how do i delete my indexes in Elasticsearch?
I need to delete the logs generated in Elasticsearch from an application I have here and leave the logs from the last 3 days, I've looked here on the forum but everything I found had no effect for me.
I got to run the command
Just so we're clear, you want to delete the indices that these logs are stored in?
yes that's right, because the logs that elastic generates are taking up a lot of disk space and I need to delete them for that, but I'll explain better.
I have an application monitored by filebeat and generating logs in elasticsearch and I would like to exclude these logs, but not exclude the files, but only the logs
about the documentation you gave me I don't know if I understood very well but let's go
GET /_cat/indices/my-index-*?v=true&s=index
this command that you have as an example would be what I have to do replacing the name of my index files?
what would be the _cat??
I ran the command and as output I got the result below so if you can help me to interpret it so I can get to the solution that would be to exclude the logs
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open .ds-filebeat-8.1.2-2022.04.03-000001 ufsu3EJcRsSNFJ0vKJzOvg 1 1 1744 0 535.3kb 535.3kb
let me see if i understand
Is that just what I have to do?
if I had another application collecting data, such as auditbeat, logstash, etc... just do the search and delete them and it will be deleted from Elasticsearch?
I understand, but if I want to do with it, delete the logs and keep the data always from the last 3 days, how do I do it? I need to make an automatic routine of this
did the delete command you gave me this message there?
{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "index [.ds-filebeat-8.1.2-2022.04.03-000001] is the write index for data stream [filebeat-8.1.2] and cannot be deleted"
}
],
"type" : "illegal_argument_exception",
"reason" : "index [.ds-filebeat-8.1.2-2022.04.03-000001] is the write index for data stream [filebeat-8.1.2] and cannot be deleted"
},
"status" : 400
}
that's exactly it, I have another server here that has more than 500GB occupied and I need to delete the logs and as I don't know how to do it, I asked for help here
The disconnect for me is that when you posted the output from _cat/indices?v, there was only one index in the output, that was 535.3kb. So where's the 500GB coming from? What other server?
hello warkolm
a thousand pardons, is that the company here changed the accesses to this server that has the 500GB of logs and I'm waiting to be released and this 535.3kb is a test environment that I'm using, but on the server that has the 500GB of logs while I I had access, I did the same procedures here, guided by you and gave the same error as the one I'm in the test environment
I managed to solve it after many tests here directly inside the kibana creating a policy, I had many doubts about this policy because I was not understanding the phases that are here and after a lot of studying these phases, now it became clearer and I managed to do it and it is working normally with this ILM policy enabled and everything automated the way I needed it, which is every 3 days to delete the logs and so I will always have 3-day logs which is what I need according to the attached print
anyway, thank you for all your attention and help.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.