Do you use SSD's? Because if not I don't know why you want to risk data loss just because of 5TB. 5TB are only 5% of the price you payed for all that storage. Just invest another 150$ and you are fine. But thats just my opinion. Maybe there is a good reason for your question.
We will reach our limit in few weeks and as I don't have more storage for now, I 'm trying to save some space with this.
Also, I will need to buy another server to increase my storage size. That implie hardware, licences, etc.. extra cost.
It's a shame to lose 1Tb per node I think but maybe I'm not right.
If so your settings should be okay but bear in mind the possible risk that comes with it. Also if you have only a few weeks left these 4,4% of extra space will give you only a couple of days more. So your investment has to be done either way. Default elasticsearch limitations are there for a reason, of course you can change them to your liking. The only problem that I see is that if you are at 99,4% full disks you will have no time to plan your further steps. If you have already planed what you will do when your disks reach that limit go for it^^.
is the standard answer of every elastic team member . I thing they train it on their first they when they join the company. But I think its just hard to create general rules which work for people using 1GB of data and some others using petabytes and supercomputers (who knows what elastic is used for). I think the logic is that if you run into these limitations that you still have time to take steps against it. Maybe if a company doesn't check their cluster state and the suddenly realise its close to full they can still increase this size and order new servers.
sounds good to me. Removing replicas is the first thing you should try. Maybe force merges will help but they will lead to longer query times and I dont think it helps that much. Snapshots are an idea. But you will need a lot of storage for them.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
. I thing they train it on their first they when they join the company. But I think its just hard to create general rules which work for people using 1GB of data and some others using petabytes and supercomputers (who knows what elastic is used for). I think the logic is that if you run into these limitations that you still have time to take steps against it. Maybe if a company doesn't check their cluster state and the suddenly realise its close to full they can still increase this size and order new servers.