MultiSearchResponse.Item is not giving expected result

Elastic Stack Elasticsearch
1

Hi,
I have a total of 10000 docs in my index A. I am trying to access response hits in my custom plugin class to modify my output for a use case.
When I am using the below for loop statement, response hits are only 9945.
My expected hits should be 10000.
for(MultiSearchResponse.Item response : items)
Attaching a

HitsSize
HitsSize.png968×425 49.7 KB
screenshot of response.
Please help.

2

Just a guess because you're not telling a lot about the context.

Are you running this in an integration test?
And you indexed 10k docs?
And just after run the search?
But you did not call refresh before searching?

If I'm wrong then share what the query is.

3

I am using MultiSearchResponse in my custom plugin class to perform/modify some output where ever certain query is matched.
I have 10000 docs. Static index.
Ex:
{ "query": { "bool" : { "must" : { "term" : { "pname" : "apple" } } } } }

4

That doesn't answer my questions.

Anyway what is the output of

GET _search?size=0

In Kibana console?

5

I executed the query

     curl -XGET localhost:9208/querytest/_search?size=0

Result:

{"took":1,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":10000,"max_score":0.0,"hits":[]}}

While debug also I face the same issue with totalHits and InternalSearchHit.
Attaching snapshot.

image
image.png1246×269 40.3 KB

Is this info relate for the question? please correct me if I'm wrong

6

Here is the original image you posted:

image
image.jpg1376×600 160 KB

The totalHits is 9945, not 10000.

I don't understand why you think there is an issue.

7

It's showing as 9945 but the total hits when I run through curl which I posted earlier
gives totalhits as 10000

image%20(1)
image (1).png1246×269 40.8 KB

In the below code :

SearchHit[] searchHits = response.getResponse().getHits().getHits();
    if(searchHits.length > 0){
      for(SearchHit hit : searchHits){
        Map<String, Object> sourceMap = hit.sourceAsMap();

I am left with 9945 hits instead of 10000 hits.

8

At the moment you initially run the request, you had 9945 hits that matches your query. That's all what I can say.
There is no bug here.

As I said, I believe that you did not refresh the index before running your first search.

But if you think there is an issue, please share the full code which can help to reproduce the problem.

9 
curl -XGET localhost:9208/querytest/_search?size=0 -d '{"query": {
"bool" : {
  "must" : {
    "term" : { "nde" : "scs" }
  }
} }}'

Result:

{"took":3,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":10000,"max_score":0.0,"hits":[]}}

I have created a new index, To confirm I have refreshed index with _refresh with curl.
Still i face same issue

10

Sorry but what is the issue? I can't see it here. Could you explain again?

11

I have an Index containing 10k docs, In every document the field nde:scs exists.
When I search this query in kibana/curl, I get the output as expected which is 10000 hits.
But when I try to access the hits through MultiSearchResponse Class in the code I am getting hits as only 9945 while debugging.

Kibana snapshot

image%20(2)
image (2).png1176×350 28.1 KB

Debug snapshot

image%20(1)
image (1).png1246×269 40.8 KB

12

I see. Let me check.

13

So I created an index with 10000 records. Then I ran the following code:

SearchRequest searchRequest = new SearchRequest("test");
searchRequest.source(new SearchSourceBuilder().size(10000));
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

System.out.println("response size = " + searchResponse.getHits().getHits().length);

MultiSearchRequest request = new MultiSearchRequest().add(searchRequest);
MultiSearchResponse response = client.msearch(request, RequestOptions.DEFAULT);

System.out.println("response size = " + response.getResponses()[0].getResponse().getHits().getHits().length);

This is giving:

response size = 10000
response size = 10000

So I'm always getting the right number of records.

What is your version? I'm using 6.6.2.

14

I am getting searchHits.length as 9945 but totalHits as 10000 can u help me understand.

image%20(3)
image (3).png1229×439 38.4 KB

Code block which I'm using for response

   for(MultiSearchResponse.Item response : items) {
        if (response.getResponse() == null) {
            break;
        }
        SearchHit[] searchHits = response.getResponse().getHits().getHits();
        if(searchHits.length > 0){
            for(SearchHit hit : searchHits){
                sourceSecondMap = hit.sourceAsMap();
15

Version 5.0.1

16

Then upgrade may be. That version is sooooo old.

I can if you share a full example as I did.
If you don't share more, it's hard to help.

17

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.