How to delete elasticsearch logs?

Elastic Stack Elasticsearch
1

good afternoon folks

how do i delete my indexes in Elasticsearch?
I need to delete the logs generated in Elasticsearch from an application I have here and leave the logs from the last 3 days, I've looked here on the forum but everything I found had no effect for me.
I got to run the command

curl -XDELETE http://my-elastic:9200/elasticsearch.log

but it gives the following message

curl(52) Empty reply from server

so I don't know if I'm doing this right, since I'm also a beginner in elastic
grateful to anyone who can help

2

Just so we're clear, you want to delete the indices that these logs are stored in?

Take a look at cat indices API | Elasticsearch Guide [8.1] | Elastic, which you can use to get a list of the indices in your cluster. From there you can delete them.

3

Thanks for the answer warkolm

Just so we're clear, you want to delete the indices that these logs are stored in?

yes that's right, because the logs that elastic generates are taking up a lot of disk space and I need to delete them for that, but I'll explain better.
I have an application monitored by filebeat and generating logs in elasticsearch and I would like to exclude these logs, but not exclude the files, but only the logs

about the documentation you gave me I don't know if I understood very well but let's go

GET /_cat/indices/my-index-*?v=true&s=index

this command that you have as an example would be what I have to do replacing the name of my index files?
what would be the _cat??

4

Just run the GET /_cat/indices?v command in Kibana Dev tools, then post that output :slight_smile:

5

perfect warklom

I ran the command and as output I got the result below so if you can help me to interpret it so I can get to the solution that would be to exclude the logs

health status index                                uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   .ds-filebeat-8.1.2-2022.04.03-000001 ufsu3EJcRsSNFJ0vKJzOvg   1   1       1744            0    535.3kb        535.3kb
6

Ok, if you run a DELETE .ds-filebeat-8.1.2-2022.04.03-000001 then those logs will be gone.

7

let me see if i understand
Is that just what I have to do?
if I had another application collecting data, such as auditbeat, logstash, etc... just do the search and delete them and it will be deleted from Elasticsearch?

8

That's one way to do it, yes.

However, what version are you running?

9

I understand, if it's the Elasticsearch version you're asking, it's the last released version 8.1

10

Then Elasticsearch will automatically delete things for you, as it uses datastreams and ILM.

You can manually delete them if you want though.

11

I understand, but if I want to do with it, delete the logs and keep the data always from the last 3 days, how do I do it? I need to make an automatic routine of this

12

something else warkolm

did the delete command you gave me this message there?

{
   "error" : {
     "root_cause" : [
       {
         "type" : "illegal_argument_exception",
         "reason" : "index [.ds-filebeat-8.1.2-2022.04.03-000001] is the write index for data stream [filebeat-8.1.2] and cannot be deleted"
       }
     ],
     "type" : "illegal_argument_exception",
     "reason" : "index [.ds-filebeat-8.1.2-2022.04.03-000001] is the write index for data stream [filebeat-8.1.2] and cannot be deleted"
   },
   "status" : 400
}
13

Ok, it's probably worth taking a step back here.

Why are you trying to delete these logs that are stored in the indices? Are you running out of space?

14

that's exactly it, I have another server here that has more than 500GB occupied and I need to delete the logs and as I don't know how to do it, I asked for help here

15

The disconnect for me is that when you posted the output from _cat/indices?v, there was only one index in the output, that was 535.3kb. So where's the 500GB coming from? What other server?

16

hello warkolm
a thousand pardons, is that the company here changed the accesses to this server that has the 500GB of logs and I'm waiting to be released and this 535.3kb is a test environment that I'm using, but on the server that has the 500GB of logs while I I had access, I did the same procedures here, guided by you and gave the same error as the one I'm in the test environment

1 Like
17

good afternoon warkolm

I managed to solve it after many tests here directly inside the kibana creating a policy, I had many doubts about this policy because I was not understanding the phases that are here and after a lot of studying these phases, now it became clearer and I managed to do it and it is working normally with this ILM policy enabled and everything automated the way I needed it, which is every 3 days to delete the logs and so I will always have 3-day logs which is what I need according to the attached print
anyway, thank you for all your attention and help.

elastic3
elastic31636×706 75.1 KB

18

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.