# Elasticsearch-sql can't query unindexed field

**URL:** <https://discuss.elastic.co/t/elasticsearch-sql-cant-query-unindexed-field/251635>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-sql\
**Created:** [October 10, 2020, 9:17am UTC](https://discuss.elastic.co/t/elasticsearch-sql-cant-query-unindexed-field/251635 "2020-10-10T09:17:14Z")\
**Posts on this page:** 8\
**Page:** 2

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 13, 2020, 10:16am UTC](https://discuss.elastic.co/t/elasticsearch-sql-cant-query-unindexed-field/251635/21 "2020-10-13T10:16:39Z")

</div>

You mentioned 50 fields above, which is considerably lower than 1000....

---

<div class="post-metadata">

**Author:** ![ITzhangqiang](https://avatars.discourse-cdn.com/v4/letter/i/2bfe46/32.png) [@ITzhangqiang](https://discuss.elastic.co/u/ITzhangqiang)\
**Post date:** [October 13, 2020, 11:14am UTC](https://discuss.elastic.co/t/elasticsearch-sql-cant-query-unindexed-field/251635/22 "2020-10-13T11:14:46Z")

</div>

50,This is just one example

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 13, 2020, 11:28am UTC](https://discuss.elastic.co/t/elasticsearch-sql-cant-query-unindexed-field/251635/23 "2020-10-13T11:28:27Z")

</div>

I suspect the main problem with fetching non-indexed fields from source is that the format has not been validated and defined at index time. You could easily have field conflicts, e.g. string vs objects, that would cause problems. In a relational database the column format is controlled even if it is not indexed, which is what the indexing step in Elasticsearch does.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 13, 2020, 1:37pm UTC](https://discuss.elastic.co/t/elasticsearch-sql-cant-query-unindexed-field/251635/24 "2020-10-13T13:37:43Z")

</div>

I normally only trust what I can measure.  
You might see a difference indeed. The question is more about "how much", when the cluster has loaded most of the data structures in the file system cache? Is that nanoseconds, milliseconds or dozen of seconds we are speaking about?

Of course, 50 fields vs 1000 fields might be another story.

---

<div class="post-metadata">

**Author:** ![bogdan.pintea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bogdan.pintea/32/45740_2.png) [@bogdan.pintea](https://discuss.elastic.co/u/bogdan.pintea)\
**Post date:** [October 13, 2020, 2:45pm UTC](https://discuss.elastic.co/t/elasticsearch-sql-cant-query-unindexed-field/251635/25 "2020-10-13T14:45:46Z")

</div>

ES/SQL will make use of the new [`fields` API](https://www.elastic.co/guide/en/elasticsearch/reference/master/search-fields.html) in one of the next coming releases. That will lift some of the inconveniences of having to selecting fields from specific "sources".

That said, however:

- as it's been pointed out, doing actual performance analysis and point out problems that way is generally more productive than theoretical discussions.
- doing `SELECT *` on hundred or more fields is rarely useful. Feel free to share your use case if you think we should improve anything here, though, we're happy to help!

---

<div class="post-metadata">

**Author:** ![ITzhangqiang](https://avatars.discourse-cdn.com/v4/letter/i/2bfe46/32.png) [@ITzhangqiang](https://discuss.elastic.co/u/ITzhangqiang)\
**Post date:** [October 14, 2020, 2:38am UTC](https://discuss.elastic.co/t/elasticsearch-sql-cant-query-unindexed-field/251635/26 "2020-10-14T02:38:34Z")

</div>

My cluster has 500TB data， Do you think most data caches are possible？

---

<div class="post-metadata">

**Author:** ![ITzhangqiang](https://avatars.discourse-cdn.com/v4/letter/i/2bfe46/32.png) [@ITzhangqiang](https://discuss.elastic.co/u/ITzhangqiang)\
**Post date:** [October 14, 2020, 2:52am UTC](https://discuss.elastic.co/t/elasticsearch-sql-cant-query-unindexed-field/251635/27 "2020-10-14T02:52:03Z")

</div>

Thankyou.  
My cluster has a large amount of data(500TB), almost every index has hundreds of fields.  
Most businesses use RestClient for queries. But now some businesses want to try SQL.  
But I found,SQL can't retrieve unindexed fields（ I have to modify all index mapping to set these fields "index:true" or "store:true",It's inconvenient）.And some indexes have thousands of fields, I find it a little slow to retrieve these fields using SQL.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 11, 2020, 2:52am UTC](https://discuss.elastic.co/t/elasticsearch-sql-cant-query-unindexed-field/251635/28 "2020-11-11T02:52:05Z")

</div>

Closed on November 11, 2020, 2:52am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/elasticsearch-sql-cant-query-unindexed-field/251635.md?page=1)
