Elasticsearch/data size reduction after restart


#1

Hello, I'm running Elasticsearch on a Docker container. I start the container mapping the folder /usr/share/elasticsearch/data on a local folder in order to retain the data even if the container is destroyed.
I have noticed that after stopping and restarting the elasticsearch container the elasticsearch/data folder size is shrinking a lot.
However, it doesn't seem to be causing data loss, because I'm still able to see all the data on Kibana.
I've compared the folder size before and after restarting elasticsearch and definitely there is something going on:

Compare before:

[...]
8.0K    ./indices/logstash-apache-2016.09.09/_state
8.0K    ./indices/logstash-apache-2016.09.09/3/_state
4.5M    ./indices/logstash-apache-2016.09.09/3/translog
2.0M    ./indices/logstash-apache-2016.09.09/3/index
6.5M    ./indices/logstash-apache-2016.09.09/3
[...]

With after elasticsearch restart:

[...]
8.0K    ./indices/logstash-apache-2016.09.09/_state
8.0K    ./indices/logstash-apache-2016.09.09/3/_state
20K     ./indices/logstash-apache-2016.09.09/3/translog
2.4M    ./indices/logstash-apache-2016.09.09/3/index
2.4M    ./indices/logstash-apache-2016.09.09/3
[...]

I would like to understand if there is something wrong I'm doing here :neutral_face:


(Mark Walkom) #2

Translog will reduce because it is emptied - ie things are played and cleared.


#3

Thanks. Is this size reduction also performed during running time? Or I should plan these container restarts?


(Mark Walkom) #4

Yes, it's constantly cleared.


#5

Thanks again :slight_smile:


(system) #6