I have to index huge volume of data frequently for research purpose. 60,000,000 docs are one of my recent task for indexing. Fortunately, the size of docs is very small, so the total size of bulk index file for 60 M docs is only 11 G. I used the following command for Solr to prevent memory error…

What is the best way for huge bulk file indexing?

Topic		Replies	Views
Indexing large number of files each with a huge size Elasticsearch	3	488	July 6, 2017
Elasticsearch bulk index Elasticsearch	5	365	July 6, 2017
Indexing Performance, Threads + Bulk Size Elasticsearch	2	409	July 6, 2017
Improving Bulk Indexing Elasticsearch	12	4586	July 6, 2017
Indexing 150 G of data Elasticsearch	2	343	July 6, 2017

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

What is the best way for huge bulk file indexing?

Related topics