Hellow guys ! My post & 1st question here, be kind to me 
So here's my problem : we have multiple indices, but the primary and bigger one is starting to lose documents by the seconds I'm talking here. (2 years of data with nearly 5-6millions documents per month).
Elasticsearch version is 5.6.4 if I remember correctly. I was doing some stuff to integrate new documents into another index, and when I checked everything I realized that I was losing like 1 doc every 1-2second litteraly. It was supposed to be a bulk insert, with only create as command so I'm pretty sure there's no problem here (This code is being used since more than 1 year, I just changed the way our ID is formatted actually and again, in other index which is getting new document as wanted)
I tried checking the task list with
GET _tasks
{
"nodes": {
"2kn_tachR7iI36EcBn__1Q": {
"name": "Sentinel-prod-ELK04",
"transport_address": "172.26.115.12:9300",
"host": "172.26.115.12",
"ip": "172.26.115.12:9300",
"roles": [
"master",
"data",
"ingest"
],
"tasks": {
"2kn_tachR7iI36EcBn__1Q:1351920880": {
"node": "2kn_tachR7iI36EcBn__1Q",
"id": 1351920880,
"type": "direct",
"action": "cluster:monitor/tasks/lists[n]",
"start_time_in_millis": 1549378356810,
"running_time_in_nanos": 84030,
"cancellable": false,
"parent_task_id": "2kn_tachR7iI36EcBn__1Q:1351920879"
},
"2kn_tachR7iI36EcBn__1Q:1351920879": {
"node": "2kn_tachR7iI36EcBn__1Q",
"id": 1351920879,
"type": "transport",
"action": "cluster:monitor/tasks/lists",
"start_time_in_millis": 1549378356810,
"running_time_in_nanos": 139082,
"cancellable": false
}
}
}
}
}
Is there something that I missed ?
Thank you for your help !
[EDIT] Or so I thought ! I realized the documents disappearing where not older than 1 year BUT the total documents since the begining hasn't changed one bit.
I divided my research in 2 groups : 1st group 'now from 1 year ago' & 2nd group '1year ago from ... 10 years ago' though we don't have data that old.
Result ? 1st group is losing doc, but 2nd group isn't gaining any, and the total doesn't bulge ...
Sooo... I'm still lost haha
