IDs query returning incomplete results

[cwinkler]

I have a set of distinct document IDs.
I know for sure that these documents exist, because the IDs have been retrieved by some previous queries.

Now I'm executing an IDs query for these IDs.
However, the first execution of such query sometimes does not return complete results, i.e., some documents missing.
When I execute the same query a second time, the results are complete.
Setting size to the number of IDs or even a way larger number has no effect on this behavior.

Why does this happen?
And how can I force ES to return full results?

[Christian_Dahlqvist]

Which version of Elasticsearch are you using?

[cwinkler]

We are currently using 5.5.3 but we saw this behavior even with 2.x

[cwinkler]

As I didn't get an answer so far, here is some more information.
The first requests use _source=false and just get the document IDs as results.

The second and problematic IDs query is meant for getting the details of some previous results using _source=true

Maybe the issue is related to that?

[Christian_Dahlqvist]

Can you perhaps show a minimal recreating script so we better can understand what the issue is?

[cwinkler]

The preparation is to store some documents with significant size (~10kB each), and then fetch those documents with the following query.
The query is logged from Java API in terms of SearchRequestBuilder.toString()

The issue seems to occur particularly when ES is freshly started, i.e., it may be related to some caching.

Invoking the query for the first time may return only ~20 results, and invoking again afterwards will give all 25 espected results.

 {
 	"size": 25,
 	"timeout": "2m",
 	"query": {
 		"bool": {
			"filter": [{
 					"ids": {
 						"type": [
 							"LogMessage"
 						],
 						"values": [
 							"01e8430a-e1b2-1e7a-aa96-c1e943736030",
 							"01e8430a-e1b1-1804-9628-d5b9e12b0a84",
 							"01e8430a-e4af-1781-92ba-cff7229bf670",
 							"01e8430a-e1b3-13d6-9628-d5b9e12b0a84",
 							"01e8430a-e1b2-1f4e-badf-b7e530d25a2a",
 							"01e8430a-e1b4-1a4c-aa96-c1e943736030",
 							"01e8430a-e1b3-1fef-92ba-cff7229bf670",
 							"01e8430a-e1b1-1fb7-badf-b7e530d25a2a",
 							"01e8430a-e1b3-15b6-aa6a-3b4497e409fe",
 							"01e8430a-e1b4-1a79-aa6a-3b4497e409fe",
 							"01e8430a-e1b3-1ae7-9628-d5b9e12b0a84",
 							"01e8430a-e1b3-13d7-aa6a-3b4497e409fe",
 							"01e8430a-e1b1-1eeb-92ba-cff7229bf670",
 							"01e8430a-e1b3-1abc-92ba-cff7229bf670",
 							"01e8430a-e1b6-1a0a-aa6a-3b4497e409fe",
 							"01e8430a-e1b4-151b-aa96-c1e943736030",
 							"01e8430a-e3bc-1126-9a01-319778ddbf4e",
 							"01e8430a-e398-1ec7-bb4a-d9ce79562946",
 							"01e8430a-e28b-1b8d-bb4a-d9ce79562946",
 							"01e8430a-e4ed-15c0-bb4a-d9ce79562946",
 							"01e8430a-e1c0-112e-aa96-c1e943736030",
 							"01e8430a-e1bf-137d-aa96-c1e943736030",
 							"01e8430a-e1b1-1ee9-aa96-c1e943736030",
 							"01e8430a-e37a-1632-ac83-ff9b055f540e",
 							"01e8430a-e1f2-1507-bb4a-d9ce79562946"
 						],
 						"boost": 1.0
 					}
 				}
 			],
 			"disable_coord": false,
 			"adjust_pure_negative": true,
 			"boost": 1.0
 		}
 	},
 	"_source": {
 		"includes": [],
 		"excludes": [
 			"activities",
 			"transitions",
 			"plugins",
 			"rawTransitions"
 		]
 	}
 }