I'm testing the indexing of 100 million documents, it took about 400GB of
the hard drive.
Is there a minimum free hard drive space needed for the index to work OK?
I'm asking because after we indexed 100 million documents we tested the
index and it worked OK,
but then when trying to optimize the optimize took days and then the index
did not respond.
The hard drive had only 10 GB free space so we tried to copy the index to a
new hard drive with a bigger free space, but the index is still not
functioning.
It will work until it's full, but then ES will fall over.
Merging does require a certain amount of disk space, usually the same
amount as the segment that is being merged as it has to take a copy of the
shard to work on. So for a 10GB segment, you'd need at least 10GB free.
How many shards do you have for the index, or how many are you trying to
optimise (merge) down to?
I'm testing the indexing of 100 million documents, it took about 400GB of
the hard drive.
Is there a minimum free hard drive space needed for the index to work OK?
I'm asking because after we indexed 100 million documents we tested the
index and it worked OK,
but then when trying to optimize the optimize took days and then the index
did not respond.
The hard drive had only 10 GB free space so we tried to copy the index to
a new hard drive with a bigger free space, but the index is still not
functioning.
When I tried to optimize the index had 51 shards.
Regards, Ophir
On Wednesday, July 2, 2014 11:27:50 AM UTC+3, Mark Walkom wrote:
It will work until it's full, but then ES will fall over.
Merging does require a certain amount of disk space, usually the same
amount as the segment that is being merged as it has to take a copy of the
shard to work on. So for a 10GB segment, you'd need at least 10GB free.
How many shards do you have for the index, or how many are you trying to
optimise (merge) down to?
On 2 July 2014 18:13, Ophir Michaeli <ophirm...@gmail.com <javascript:>>
wrote:
Hi all,
I'm testing the indexing of 100 million documents, it took about 400GB of
the hard drive.
Is there a minimum free hard drive space needed for the index to work OK?
I'm asking because after we indexed 100 million documents we tested the
index and it worked OK,
but then when trying to optimize the optimize took days and then the
index did not respond.
The hard drive had only 10 GB free space so we tried to copy the index to
a new hard drive with a bigger free space, but the index is still not
functioning.
When I tried to optimize the index had 51 shards.
Regards, Ophir
On Wednesday, July 2, 2014 11:27:50 AM UTC+3, Mark Walkom wrote:
It will work until it's full, but then ES will fall over.
Merging does require a certain amount of disk space, usually the same
amount as the segment that is being merged as it has to take a copy of the
shard to work on. So for a 10GB segment, you'd need at least 10GB free.
How many shards do you have for the index, or how many are you trying to
optimise (merge) down to?
I'm testing the indexing of 100 million documents, it took about 400GB
of the hard drive.
Is there a minimum free hard drive space needed for the index to work OK?
I'm asking because after we indexed 100 million documents we tested the
index and it worked OK,
but then when trying to optimize the optimize took days and then the
index did not respond.
The hard drive had only 10 GB free space so we tried to copy the index
to a new hard drive with a bigger free space, but the index is still not
functioning.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.