# How can I search for the latest data entered in the indexes? **URL:** \ **Category:** Elasticsearch\ **Created:** [December 26, 2023, 3:38pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972 "2023-12-26T15:38:22Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![deep1](https://avatars.discourse-cdn.com/v4/letter/d/43a26b/32.png) [@deep1](https://discuss.elastic.co/u/deep1)\ **Post date:** [December 26, 2023, 3:38pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/1 "2023-12-26T15:38:22Z")
For example, I want to search in 100,000 documents from each index, and it is not possible to add to that, and they are first loaded into the cache, then only this data is searched ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 26, 2023, 3:54pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/2 "2023-12-26T15:54:20Z")
If you have a timestamp field, you can filter on that field to search only in the now/m-15m documents. ---
**Author:** ![deep1](https://avatars.discourse-cdn.com/v4/letter/d/43a26b/32.png) [@deep1](https://discuss.elastic.co/u/deep1)\ **Post date:** [December 26, 2023, 4:49pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/3 "2023-12-26T16:49:14Z")
Yes, I have a timestamp field. But it does not solve the problem, I want the search to be on the latest data only, meaning that the search is always on the latest data, for example, I have a lot of data in the hard drive and I want only part of this data to be uploaded to the cache automatically and then searched for it, but the rest of the data remains in the hard drive But I want that to be taking into account that new data is always entered ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 26, 2023, 5:00pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/4 "2023-12-26T17:00:21Z")
What "cache" are you talking about? ---
**Author:** ![deep1](https://avatars.discourse-cdn.com/v4/letter/d/43a26b/32.png) [@deep1](https://discuss.elastic.co/u/deep1)\ **Post date:** [December 26, 2023, 5:02pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/5 "2023-12-26T17:02:15Z")
RAM memory ---
**Author:** ![deep1](https://avatars.discourse-cdn.com/v4/letter/d/43a26b/32.png) [@deep1](https://discuss.elastic.co/u/deep1)\ **Post date:** [December 26, 2023, 5:34pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/6 "2023-12-26T17:34:07Z")
i want execute query in last 100000 doc that insert to database in all time if query does not match any doc in last 100000 doc return null ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 27, 2023, 3:06am UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/7 "2023-12-27T03:06:11Z")
I'm not sure you really can do that only on the last 100000 last docs, unless you have a number which tells the position of the document. But I'm not sure why would you need to do that. Elasticsearch can search within millions of docs without any problem. I don't understand the use case. ---
**Author:** ![deep1](https://avatars.discourse-cdn.com/v4/letter/d/43a26b/32.png) [@deep1](https://discuss.elastic.co/u/deep1)\ **Post date:** [December 27, 2023, 6:19am UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/8 "2023-12-27T06:19:58Z")
Because I don't have a large cache to process a large number so I want to do hashing so that the processing is faster ---
**Author:** ![deep1](https://avatars.discourse-cdn.com/v4/letter/d/43a26b/32.png) [@deep1](https://discuss.elastic.co/u/deep1)\ **Post date:** [December 27, 2023, 6:22am UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/9 "2023-12-27T06:22:07Z")
Is there a similar way or approach to what I want, for example, specific settings or a specific code close to my request, such as merging the timestamp field with something else so that it is done better ---
**Author:** ![deep1](https://avatars.discourse-cdn.com/v4/letter/d/43a26b/32.png) [@deep1](https://discuss.elastic.co/u/deep1)\ **Post date:** [December 27, 2023, 6:30am UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/10 "2023-12-27T06:30:51Z")
Another reason is that I don't want to be searched in a larger range than I want. For example, I want the search to be in 100,000 documents, so I don't want it to go beyond that if the result matches, unless the result is that there is no result ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 27, 2023, 9:48am UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/11 "2023-12-27T09:48:50Z")
What makes you think that Elasticsearch won't be efficient? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 27, 2023, 9:50am UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/12 "2023-12-27T09:50:51Z")
I still don't understand the business reason for your use case. I think that you are trying to solve a problem which actually does not exist. Do you see any memory error in Elasticsearch logs? If so, please share what you are seeing. ---
**Author:** ![deep1](https://avatars.discourse-cdn.com/v4/letter/d/43a26b/32.png) [@deep1](https://discuss.elastic.co/u/deep1)\ **Post date:** [December 27, 2023, 2:41pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/13 "2023-12-27T14:41:24Z")
Because I have a lot of data if I query it at once it will take some time and I don't want to be late I want the query to be very fast Therefore, I would like to limit the search to part of the data, which is only recent data ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 27, 2023, 2:45pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/14 "2023-12-27T14:45:39Z")
How many indices and shards are you querying? What is the size of these indices and shards? What is the size and specification of the cluster? What latencies are you seeing when you query the data? ---
**Author:** ![deep1](https://avatars.discourse-cdn.com/v4/letter/d/43a26b/32.png) [@deep1](https://discuss.elastic.co/u/deep1)\ **Post date:** [December 27, 2023, 2:48pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/15 "2023-12-27T14:48:07Z")
No, there is nothing wrong I'll explain to you. For example, if you used the timestamp field and had it return to me the last 100,000 stored documents Well, he will search the indexes until all the required data is collected But here he's going to search all the data, and I don't want that. I want him to only look at the extent that I specify and not exceed it if he finds the matching data in the first 100,000 documents he returns the result otherwise he returns blank ---
**Author:** ![deep1](https://avatars.discourse-cdn.com/v4/letter/d/43a26b/32.png) [@deep1](https://discuss.elastic.co/u/deep1)\ **Post date:** [December 27, 2023, 3:20pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/16 "2023-12-27T15:20:18Z")
The times are big for example if you search for 10000 documents the time is 6000ms and more and sometimes less by a small difference As for the number of indicators and parts, I do not know how many they are precisely. But I am looking in only one index and I think that the number of its parts does not exceed 2 ---
**Author:** ![deep1](https://avatars.discourse-cdn.com/v4/letter/d/43a26b/32.png) [@deep1](https://discuss.elastic.co/u/deep1)\ **Post date:** [December 27, 2023, 3:26pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/17 "2023-12-27T15:26:41Z")
To explain to you more If I all have 1,000,000 documents and I don't want these documents to be searched in full, I want to search only a part of them, for example, 10,000 documents When I want to inquire about 1000 documents, I do not want him to search in 1000000 documents, but I want him to search in 10000 documents only and not exceed it if he finds a matching result he returns it and if he does not find identical data in 10000 he returns null ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 27, 2023, 5:32pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/18 "2023-12-27T17:32:15Z")
You have a very big problem if it takes 6s to search for 10000 docs. I'm not speaking about extracting 10000 documents which is another story. You need to be precise here. Please share what your query looks like and what are the first 10 lines of the response. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 24, 2024, 5:32pm UTC](https://discuss.elastic.co/t/how-can-i-search-for-the-latest-data-entered-in-the-indexes/349972/19 "2024-01-24T17:32:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.