OK, so I reindexed and then collected all kinds of data before and after restarting the service.
_cat/indices output was simply
health status index pri rep docs.count docs.deleted store.size pri.store.size
yellow open myindex 5 1 96414689 0 97gb 97gb
before and:
health status index pri rep docs.count docs.deleted store.size pri.store.size
yellow open myindex 5 1 96414689 0 69.9gb 69.9gb
afterwards
As you can see, I have the default settings, so five primary nodes in total.
For each of these, the directory
/var/lib/elasticsearch/mycluster/nodes/0/indices/myindex/0/index
shows a du -sh of 20 GB before, and 16 GB afterwards.
translog folders are very small in both cases (around 20 KB).
So I ran
du /var/lib/elasticsearch/mycluster/nodes/0/indices/myindex/0/index/* | sort -n
on each of the nodes before and afterwards to get an idea of which files change in size.
After the restart, the files in this directory either stayed constant in size, or they completely disappeared.
The largest of the disappeared files for a sample directory were (first column is size in KByte):
284632 /var/lib/elasticsearch/mycluster/nodes/0/indices/myindex/0/index/_t0.cfs
370488 /var/lib/elasticsearch/mycluster/nodes/0/indices/myindex/0/index/_3v.nvd
477448 /var/lib/elasticsearch/mycluster/nodes/0/indices/myindex/0/index/_wy.cfs
506380 /var/lib/elasticsearch/mycluster/nodes/0/indices/myindex/0/index/_hl.cfs
608492 /var/lib/elasticsearch/mycluster/nodes/0/indices/myindex/0/index/_pc.cfs
820228 /var/lib/elasticsearch/mycluster/nodes/0/indices/myindex/0/index/_jt.nvd
838104 /var/lib/elasticsearch/mycluster/nodes/0/indices/myindex/0/index/_rl.cfs
1269064 /var/lib/elasticsearch/mycluster/nodes/0/indices/myindex/0/index/_mm.nvd
Can you roughly tell me what happens there?
I can also email you a complete list of the files if you like.
Thanks!