Elasticsearch index is deleted from application but not removed from disk

Elastic Stack Elasticsearch
1

I am using elasticsearch version 8.12.0

i deleted some indices from my cluster, but when i checked node stats from stack monitoring tab from kibana what is found Free disk is 1TB and Data is 2.2TB but total disk size is
6TB.
So i was wondering where is this extra space is being used. I was going through the directory structure used for storing the data in elasticsearch, what i found that there were directories created with name that matches the uuid of index. then i tried the cross check it with the output from api

GET /_cat/indices?v&h=index,uuid

what i found that some of the uuid is available in output and some of the uuid is not.So i am guessing that might be uuid of the indices that i have deleted earlier.

So my queries are:

1- Is my understanding is right here about the directory structure?
2- If my understanding is right then what should i do now?

Please help.

2

There will be some special/hidden (and likely small) indices not shown in your output.

How did you delete your index? Was the deletion successfully acknowledged ?

3

Can you share some screenshot of this screen?

Is the path for the elasticsearch only used by elasticsearch or you have other things on the same server using the same disk?

The space used/free is related to the disk, not just elasticsearch data, you may have other things using that space.

4

Hi @kuldeep_gupta ,

How did you delete that index in your application?

5

hi @leandrojmp

These were not the indices which i deleted earlier. but they were taking lots of space like aroung 20GB per directory and lots of directories were there.
And one more thing that these directories were found only in hot nodes (my cluster has hot,warm data nodes only).

so i deleted these directories and my cluster is working smoothly with no data loss.

What i am guessing that there is the data which was migrated from hot to warm nodes but now deleted from disk in hot nodes.

I have no other application running because mounted point is dedicated to only for storage of elasticsearch application.

And sorry i can't share screenshot because it is fixed now.No disk stats mismatch.

6

I deleted indices using delete api, and yes it was acknowledged.
Hidden indices can't have the size of 20-40 GB.

7

usind delete api

8

Of course not, but they do have a UUID you could have tried to match to your directory names. But you seem to have figured it out now to your own satisfaction.

9

Hello,
Yes i had matched the uuid with directory name.