# Long search time with mget

**URL:** <https://discuss.elastic.co/t/long-search-time-with-mget/277062>\
**Category:** Elasticsearch\
**Created:** [June 25, 2021, 1:24pm UTC](https://discuss.elastic.co/t/long-search-time-with-mget/277062 "2021-06-25T13:24:59Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![TheWorkingDeveloper](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theworkingdeveloper/32/90834_2.png) [@TheWorkingDeveloper](https://discuss.elastic.co/u/TheWorkingDeveloper)\
**Post date:** [June 25, 2021, 1:25pm UTC](https://discuss.elastic.co/t/long-search-time-with-mget/277062/1 "2021-06-25T13:25:00Z")

</div>

I currently have a single node with SSD running an elastic cluster.  
4B records split over 25 primary shards in a single index (no replication).  
The ID of every record has been manually set as it is unique and known beforehand by both the indexer and searcher since I was suggested that directly fetching with the ID would get the best performance. But all data must be searched every time, so no hot/cold/rolling index is available.

To make an example of this, the md5 checksum of a file (as ID) and the document contains tags and metadata about that document - just to paint a picture

Searching for 1,000 - 2,000 id's at a time takes around 4-8 seconds (respectively) which I believe to be fairly slow but am not sure what the exact bottleneck is. The primary thing I'd like to eliminate is that using mget with predefined ID's is the root cause of this and that searching with a keyword for the md5 checksum of a file would be faster (or any other method).

And if the issue is hardware, upgrading which component would yield the biggest performance increase and why? The CPU util is very low overall but disk read/write is often 500-600MB/s

Hardware:  
R710 system with: 2x Intel(R) Xeon(R) CPU X5680 @ 3.33GHz  
32GB DDR3 RAM with 24GB allocated to Elasticsearch  
2x2TB SSD (RAID 0)

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 28, 2021, 1:52am UTC](https://discuss.elastic.co/t/long-search-time-with-mget/277062/2 "2021-06-28T01:52:42Z")

</div>

Just so I am clear, are you doing an `_mget indexname/id` for the whole 1-2000 records?

---

<div class="post-metadata">

**Author:** ![TheWorkingDeveloper](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theworkingdeveloper/32/90834_2.png) [@TheWorkingDeveloper](https://discuss.elastic.co/u/TheWorkingDeveloper)\
**Post date:** [July 5, 2021, 8:07am UTC](https://discuss.elastic.co/t/long-search-time-with-mget/277062/4 "2021-07-05T08:07:54Z")

</div>

```auto
GET /_mget
{
  "docs": [
    {
      "_index": "my-index-000001",
      "_id": "1"
    },
    {
      "_index": "my-index-000001",
      "_id": "2"
    }
  ]
}

```

More like this, all the same index, every ID specified

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 2, 2021, 8:07am UTC](https://discuss.elastic.co/t/long-search-time-with-mget/277062/5 "2021-08-02T08:07:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.