# Querying on large docs

**URL:** https://discuss.elastic.co/t/querying-on-large-docs/341759
**Category:** Elasticsearch
**Created:** [August 27, 2023, 6:13pm UTC](https://discuss.elastic.co/t/querying-on-large-docs/341759 "2023-08-27T18:13:28Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![m4kkur0](https://avatars.discourse-cdn.com/v4/letter/m/96bed5/32.png) [@m4kkur0](https://discuss.elastic.co/u/m4kkur0)
#### Post date: [August 27, 2023, 6:13pm UTC](https://discuss.elastic.co/t/querying-on-large-docs/341759/1 "2023-08-27T18:13:28Z")

</div>

Hello all,  
I would like to know what are some good options to query an index that each doc in it structured like:  
field1, keyword  
field2, long  
field3, object, enabled: false (mostly below 1 mb but sometimes goes up to 50 mb)

what I want to achieve is,

1. query by field1 and field2 to retrieve field3. This one retrieves only 1 doc
2. query by field1 and return only field2 value. This returns multiple docs

When I tried the second option, it took too much time even if I exclude field3 from \_source in the query  
So, would using docvalue\_fields be a solution for second option? Would it still take too much heap memory because of field3?  
What do you suggest for option 1?

---

<div class="post-metadata">

### Author: ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)
#### Post date: [August 29, 2023, 10:54am UTC](https://discuss.elastic.co/t/querying-on-large-docs/341759/2 "2023-08-29T10:54:08Z")

</div>

Hi @m4kkur0,

Welcome to the community! How much time is too much time for your first query?

Can you share the queries you are running? It might also be useful to run the queries through the [Search Profiler](https://www.elastic.co/guide/en/kibana/current/xpack-profiler.html) or the [Profiler API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-profile.html) to see how long each query stage is taking.

---

<div class="post-metadata">

### Author: ![m4kkur0](https://avatars.discourse-cdn.com/v4/letter/m/96bed5/32.png) [@m4kkur0](https://discuss.elastic.co/u/m4kkur0)
#### Post date: [August 29, 2023, 11:33am UTC](https://discuss.elastic.co/t/querying-on-large-docs/341759/3 "2023-08-29T11:33:50Z")

</div>

Hello and thank you for your reply!  
my first query is

size: 1  
query:  
bool: must:  
[  
term: field1: any keyword value,  
match: field2: any long value  
]

second query

size: 10000  
query:  
term: field2: any keyword value  
\_source: [field1]

and also query 3 which I forgot to mention about it.  
In this one, we also retrieve the field 4 which is a keyword field.

size: 10000  
query: bool: must  
[  
range: field1: gte to a long value and lte to a long val,  
term: field2: a keyword value  
\_source :[field4, field2]  
]

this query runs pretty fast actually (200 to 300 ms), how much time it takes is not a problem. What concerns me is the memory usage and errors that may occur with memory usage.

For the second and third queries, it takes more than 30 seconds and sometimes node disconnected exception occurs. So I tried setting the \_source to false and using the docvalue\_fields for this two queries, then it takes about 200 ms. But what concerns me, again, memory usage.

Is using the docvalue\_field solves node disconnected errors in this case? Is it way more lighter than using \_source?

Sorry, sending this from my mobile so shortened queries and errors may be in my text

---

<div class="post-metadata">

### Author: ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)
#### Post date: [August 29, 2023, 12:43pm UTC](https://discuss.elastic.co/t/querying-on-large-docs/341759/4 "2023-08-29T12:43:27Z")

</div>

Thanks for confirming @m4kkur0. If memory usage is your main concern I would recommend using `docvalue_fields`. There is a bit more detail in [this section](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-fields.html#field-retrieval-methods) of the documentation that discusses the loading of the document when using the `_source` attribute.

With `_source` usage the entire Lucene document will be loaded:

> A document’s `_source` is stored as a single field in Lucene. This structure means that the whole `_source` object must be loaded and parsed even if you’re only requesting part of it.

---

<div class="post-metadata">

### Author: ![m4kkur0](https://avatars.discourse-cdn.com/v4/letter/m/96bed5/32.png) [@m4kkur0](https://discuss.elastic.co/u/m4kkur0)
#### Post date: [August 29, 2023, 12:53pm UTC](https://discuss.elastic.co/t/querying-on-large-docs/341759/5 "2023-08-29T12:53:15Z")

</div>

Thanks!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 26, 2023, 12:53pm UTC](https://discuss.elastic.co/t/querying-on-large-docs/341759/6 "2023-09-26T12:53:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
