I don't get the same results with Java API and Kibana

Elastic Stack Elasticsearch
1

Hello everyone,

Here is what I have:
I have an index in elasticseach storing thousands of documents. Each document includes a field called 'deviceName'.

Here is what I would like to do with java api:
I want to get all devices stored in my index and their occurence (number of time a device appears in this index).

Here is my java code:

String[] indexVector = new String[1];
     indexVector[0] = "myIndex";
     String [] values = new String[2];
    values[0] = "sum_delta";
    values[1] = "count_frames";
    String [] totalPercentage = new String[1];
    totalPercentage[0] = "devices>per_percentage";
    SearchResponse response = (SearchResponse) client.prepareSearch(indexVector)
            .setSize(10000)
            .addAggregation(AggregationBuilders.terms("devices").field("deviceName")
                    .size(100)
                    .subAggregation(AggregationBuilders.sum("sum_delta").field("delta"))
                    .subAggregation(AggregationBuilders.count("count_frames").field("cnt"))
                    .subAggregation(PipelineAggregatorBuilders.bucketScript("per_percentage")
                            .setBucketsPaths(values)
                            .script(new Script("_value0 / (_value0 + _value1 + 1)"))                
                             )
                    )
            .addAggregation(PipelineAggregatorBuilders.avgBucket("avg_per_all_devices")
                    .setBucketsPaths(totalPercentage)
                    )           
            .execute()
            .actionGet();

How I search in Kibana:

Capture.PNG1678×117 8.19 KB

Here is my results:
With java API I get 63 devices and with kibana 64 devices. However, the biggest problem for me is the occurence. Indeed, for a device that has a occurence lower than 30, i have the same result with Java api and Kibana. But, when the occurence is larger, kibana returns a occurence greater than the one returned by java API. Example: kibana returns 1010 and in the same time java api returns 880.

I really don't understand why there is so much difference.

Question:
So, please, can you tell me what is wrong in my code and what I have to do ?

Thank you for your attention and your help.

2

In the SearchResponse json in your java code, check sum_other_doc_count for your aggregation query. I think it is the number of documents which Elasticsearch has ignored while calculating your aggregations or on which aggregations were not calculated. Elasticsearch aggregations are not precise. They are approximate.

I am not sure if Kibana is using some other API exposed by elasticsearch to get better results.