Deleting parts of data from an index


(Kavya Sabu) #1

Suppose i have thousands of log lines. Would deleting some of the log lines erase the data completely from disk space or does it just not display in search results and is still present in the disk?


(Nik Everett) #2

Both. At first its just that the documents are marked as deleted. They are
later removed when the segment they live in is merged with another segment.
Updates are the same way.

Usually this works fine. If you have data that "rolls away" its usually
better to make an index per time period and then nuke the whole index.
There is a tradeoff between the number of indexes and the ease of deleting
the whole index. That is a balancing act that you have to experiment with.


(Kavya Sabu) #3

Thanks! Will research more about it.


(Isabel Drost-Fromm) #4

If you need some starting points for your research:

https://www.elastic.co/guide/en/elasticsearch/guide/current/time-based.html

https://www.elastic.co/guide/en/elasticsearch/guide/current/retiring-data.html


(Kavya Sabu) #5

Thanks for the links!


(system) #6