# Search query total hits not adding up when using multiple queries **URL:** \ **Category:** Elasticsearch\ **Created:** [July 26, 2020, 8:17pm UTC](https://discuss.elastic.co/t/search-query-total-hits-not-adding-up-when-using-multiple-queries/242676 "2020-07-26T20:17:18Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![antonsamper](https://avatars.discourse-cdn.com/v4/letter/a/9e8a1a/32.png) [@antonsamper](https://discuss.elastic.co/u/antonsamper)\ **Post date:** [July 26, 2020, 8:17pm UTC](https://discuss.elastic.co/t/search-query-total-hits-not-adding-up-when-using-multiple-queries/242676/1 "2020-07-26T20:17:18Z")
I have encountered a strange problem whereby the total hits don't seem to be adding up when I use multiple filters. I have an index with around 40 million documents. A typical document looks like something like this: ```auto { "postcodePrefix": "BD1", "transactions": [ { "price": 100000, "category": "A", "date": "2020-01-10" }, { "price": 200000, "category": "B", "date": "2020-01-20" } ] } ``` When I do simple query, for example: ```auto POST /INDEXNAME/_search?size=0 { "query": { "bool": { "must": [ { "match": { "postcodePrefix.keyword": "BD1" } } ] } } } ``` The total hits comes back as **2766** (so far so good) If I then add an additional nested query, for example: ```auto POST /INDEXNAME/_search?size=0 { "query": { "bool": { "must": [ { "nested": { "path": "transactions", "query": { "bool": { "should": [ { "match": { "transactions.category": "A" } } ] } } } }, { "match": { "postcodePrefix.keyword": "BD1" } } ] } } } ``` The total hits now comes back as **2230** (no problem yet) Last but not least, if i run the nested query again but this time with category "B", the total hits now comes back as **705** (Please note that in the data there are only 2 possible categories, "A" and "B") What i don't understand is how the following total hits are calculated: - No category specified: 2766 hits - Category A: 2230 hits - Category B: 705 hits Shouldn't the query where no category is specified return 2230 + 705 = **2935** hits? ---
**Author:** ![Vinayak\_Sapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinayak_sapre/32/45939_2.png) [@Vinayak\_Sapre](https://discuss.elastic.co/u/Vinayak_Sapre)\ **Post date:** [July 27, 2020, 3:42am UTC](https://discuss.elastic.co/t/search-query-total-hits-not-adding-up-when-using-multiple-queries/242676/2 "2020-07-27T03:42:59Z")
Easiest way to solve is run query that excludes documents with ("postcodePrefix.keyword": "BD1") AND ("transactions.category": "A" OR "transactions.category": "B") and see what you get. ---
**Author:** ![antonsamper](https://avatars.discourse-cdn.com/v4/letter/a/9e8a1a/32.png) [@antonsamper](https://discuss.elastic.co/u/antonsamper)\ **Post date:** [July 27, 2020, 8:08am UTC](https://discuss.elastic.co/t/search-query-total-hits-not-adding-up-when-using-multiple-queries/242676/3 "2020-07-27T08:08:06Z")
@Vinayak_Sapre That would just return the total hits for everything else. Can you expand on your suggestion? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [July 27, 2020, 8:32am UTC](https://discuss.elastic.co/t/search-query-total-hits-not-adding-up-when-using-multiple-queries/242676/4 "2020-07-27T08:32:53Z")
> [@antonsamper](#): > > - Category A: 2230 hits > - Category B: 705 hits If you have a document with 2 nested objects -one is A and one is B, then that single document is counted in both of the above queries. The sets overlap. ---
**Author:** ![antonsamper](https://avatars.discourse-cdn.com/v4/letter/a/9e8a1a/32.png) [@antonsamper](https://discuss.elastic.co/u/antonsamper)\ **Post date:** [July 27, 2020, 9:32am UTC](https://discuss.elastic.co/t/search-query-total-hits-not-adding-up-when-using-multiple-queries/242676/5 "2020-07-27T09:32:10Z")
Thanks @Mark_Harwood I think I understand now. So in my case, on the typical document example above, Elasticsearch see's that as 3 documents and therefore when I query with just a `postcodePrefix` it returns 1 hit but if I add an additional query with category "A" then it returns 2 hits, the root document + one of the nested documents. Is that right? Whats important in my case is the transactions count. Is there a way around this so that in this case the count only returns 1 instead of 2? like a query within a query? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [July 27, 2020, 10:02am UTC](https://discuss.elastic.co/t/search-query-total-hits-not-adding-up-when-using-multiple-queries/242676/6 "2020-07-27T10:02:07Z")
> [@antonsamper](#): > > Elasticsearch see's that as 3 documents To be accurate, elasticsearch sees it as one document that falls into multiple sets. Let's take this web page as an example document - it might have these properties. ``` { "date" : "27/7/20", "contributors" : ["mark", "anton", "vinayak"] ... } ``` So this document belongs to both your set of topics and mine. > [@antonsamper](#): > > but if I add an additional query with category "A" then it returns 2 hits, the root document + one of the nested documents. Is that right? No, I don't think so. When you say "additional query" it depends on if you mean as a Boolean AND or an OR (narrowing or expanding the criteria). If you expand then you might get more hits but if your criteria is another MUST it will narrow and you'll only ever get the same or fewer hits than the first query. ---
**Author:** ![antonsamper](https://avatars.discourse-cdn.com/v4/letter/a/9e8a1a/32.png) [@antonsamper](https://discuss.elastic.co/u/antonsamper)\ **Post date:** [July 27, 2020, 10:58am UTC](https://discuss.elastic.co/t/search-query-total-hits-not-adding-up-when-using-multiple-queries/242676/7 "2020-07-27T10:58:10Z")
@Mark_Harwood when I mentioned additional query i meant that there were 2 conditions inside the must array: ```auto { "query": { "bool": { "must": [ // condition 1 - must match postcodePrefix // condition 2 - must match category A ] } } } ``` Is the above syntax correct? I read that as an AND therefore narrowing my search ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [July 27, 2020, 11:03am UTC](https://discuss.elastic.co/t/search-query-total-hits-not-adding-up-when-using-multiple-queries/242676/8 "2020-07-27T11:03:21Z")
Correct. That should narrow. Can you post the docs/query/mappings that demonstrate an expansion? That part doesn't make sense seeing the counts go up with more MUSTs. ---
**Author:** ![antonsamper](https://avatars.discourse-cdn.com/v4/letter/a/9e8a1a/32.png) [@antonsamper](https://discuss.elastic.co/u/antonsamper)\ **Post date:** [July 28, 2020, 11:24am UTC](https://discuss.elastic.co/t/search-query-total-hits-not-adding-up-when-using-multiple-queries/242676/9 "2020-07-28T11:24:57Z")
@Mark_Harwood i think I've been thinking about my search queries the wrong way and what you mentioned above regarding the overlap now makes sense. My first query where i don't specify a category is wrong because thats just returning the number of documents where the postcodePrefix matches but thats not actually what i want. I need to know the number of transactions inside of that document i.e. the inner\_hits count. I'm going to explore inner\_hits further but I'm now also questioning whether i should modify my mapping and not have the transactions as nested documents. PS. i saw your video on " Wrestling Techniques Using Elasticsearch's Aggregations" and coincidentally I'm also working the UK Price Paid Data set. ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [July 28, 2020, 1:13pm UTC](https://discuss.elastic.co/t/search-query-total-hits-not-adding-up-when-using-multiple-queries/242676/10 "2020-07-28T13:13:50Z")
> [@antonsamper](#): > > what you mentioned above regarding the overlap now makes sense. Good to hear. > [@antonsamper](#): > > coincidentally I'm also working the UK Price Paid Data set. > [@antonsamper](#): > > I'm now also questioning whether i should modify my mapping and not have the transactions as nested documents. It made sense for my analysis at least to have transactions as their own documents and include the postcode as a term. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 25, 2020, 1:17pm UTC](https://discuss.elastic.co/t/search-query-total-hits-not-adding-up-when-using-multiple-queries/242676/11 "2020-08-25T13:17:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.