Need help with performance

Hey guys, i have been trying to make some improvements on my elasticsearch to make my graylog answer quickly with searchs and queries, but i'm having troubles with my index and shards sizes. I saw the video and the old and new documentantion and couldn't really grab the concept.
So, i have two nodes with 32 GB RAM, 62 GB of Heap and 500 GB of Disk in both nodes. We are using 50 index with 4 shards, each index gather 30M documents with and average of 12GB. Our objective is to retain the maximum logs possible, respecting the 75% disk space usage limit.
With all that said, should i have less index and more shards? I'm really lost and in need of help to make this performance improvement.

I appreciate for all of you reading and taking your time with this topic, have a great day everybody!
Based on this data,

I recommend you check out my video on installing the Elastic Stack. I cover hardware sizing and the reason why in the first part of the video.

0003_es_install

You also don't mention what kind of storage you are using. SSDs or HDDs? RAID? Local or network attached? I do a deep dive on the best storage for Elasticsearch in this video. (TL;DR use local SSDs... PERIOD!)

0001_es_storage

Rob

GitHub YouTube LinkedIn
How to install Elasticsearch & Kibana on Ubuntu - incl. hardware recommendations
What is the best storage technology for Elasticsearch?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.