# How to accelerate the searching if there is a large number of data in es?

**URL:** <https://discuss.elastic.co/t/how-to-accelerate-the-searching-if-there-is-a-large-number-of-data-in-es/101097>\
**Category:** Elasticsearch\
**Created:** [September 20, 2017, 4:05am UTC](https://discuss.elastic.co/t/how-to-accelerate-the-searching-if-there-is-a-large-number-of-data-in-es/101097 "2017-09-20T04:05:27Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![KeithTt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keithtt/32/29447_2.png) [@KeithTt](https://discuss.elastic.co/u/KeithTt)\
**Post date:** [September 20, 2017, 4:05am UTC](https://discuss.elastic.co/t/how-to-accelerate-the-searching-if-there-is-a-large-number-of-data-in-es/101097/1 "2017-09-20T04:05:27Z")

</div>

ELK version: 5.5.1

There are 5 nodes in my es cluster.

I use elk to collect nginx logs, and there is a project generate about 100GB logs every day, and it is very slow to open the dashboard of this project in kibana...

So, how to make it faster to open the dashboard? Should I add some es nodes or replace the harddisk with ssd or do something else?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 21, 2017, 2:05am UTC](https://discuss.elastic.co/t/how-to-accelerate-the-searching-if-there-is-a-large-number-of-data-in-es/101097/2 "2017-09-21T02:05:36Z")

</div>

FYI we’ve renamed ELK to the Elastic Stack, otherwise Beats feels left out 😉

> [@KeithTt](#):
>
> Should I add some es nodes or replace the harddisk with ssd or do something else?

Either of those would help. As would upgrading to 5.6.1.

However it'd help if you provided your node size, the number of indices and shards, OS and JVM version.

---

<div class="post-metadata">

**Author:** ![KeithTt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keithtt/32/29447_2.png) [@KeithTt](https://discuss.elastic.co/u/KeithTt)\
**Post date:** [September 22, 2017, 6:35am UTC](https://discuss.elastic.co/t/how-to-accelerate-the-searching-if-there-is-a-large-number-of-data-in-es/101097/3 "2017-09-22T06:35:55Z")

</div>

```auto
# cat /etc/issue
CentOS release 6.6 (Final)
Kernel \r on an \m

```

```auto
# java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)

```

62G mem

```auto
# free -g
             total used free shared buffers cached
Mem: 62 61 0 0 0 13
-/+ buffers/cache: 48 14
Swap: 0 0 0

```

1T disk

```auto
# df -hT
Filesystem Type Size Used Avail Use% Mounted on
/dev/sda2 ext4 40G 12G 26G 31% /
tmpfs tmpfs 32G 0 32G 0% /dev/shm
/dev/sda1 ext4 190M 57M 124M 32% /boot
/dev/sda5 ext4 128G 7.3G 114G 6% /nh
/dev/sdb ext4 917G 635G 235G 73% /data

```

Indices: 759

Total Shards: 6572

😭

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 22, 2017, 6:41am UTC](https://discuss.elastic.co/t/how-to-accelerate-the-searching-if-there-is-a-large-number-of-data-in-es/101097/4 "2017-09-22T06:41:17Z")

</div>

You have a very high shard count. Can you use `_shrink` on some of them?

---

<div class="post-metadata">

**Author:** ![KeithTt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keithtt/32/29447_2.png) [@KeithTt](https://discuss.elastic.co/u/KeithTt)\
**Post date:** [September 22, 2017, 7:28am UTC](https://discuss.elastic.co/t/how-to-accelerate-the-searching-if-there-is-a-large-number-of-data-in-es/101097/5 "2017-09-22T07:28:15Z")

</div>

how to use shrink ...

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 22, 2017, 7:29am UTC](https://discuss.elastic.co/t/how-to-accelerate-the-searching-if-there-is-a-large-number-of-data-in-es/101097/6 "2017-09-22T07:29:55Z")

</div>

[https://www.elastic.co/guide/en/elasticsearch/reference/5.6/indices-shrink-index.html](https://www.elastic.co/guide/en/elasticsearch/reference/5.6/indices-shrink-index.html) 🙂

---

<div class="post-metadata">

**Author:** ![KeithTt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keithtt/32/29447_2.png) [@KeithTt](https://discuss.elastic.co/u/KeithTt)\
**Post date:** [September 22, 2017, 7:32am UTC](https://discuss.elastic.co/t/how-to-accelerate-the-searching-if-there-is-a-large-number-of-data-in-es/101097/7 "2017-09-22T07:32:40Z")

</div>

thanx a lot, let me read it and have a try. 😘

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 20, 2017, 7:32am UTC](https://discuss.elastic.co/t/how-to-accelerate-the-searching-if-there-is-a-large-number-of-data-in-es/101097/8 "2017-10-20T07:32:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.