Elasticsearch File Indexing tuning


(ravikanth) #1

Hi All,
Iam new to elasticsearch and using elasticsearch-1.0.0.RC1.
when iam indexing the bulk files(documents like txt,pdf etc) it
taking more time.
Apart from my application have one table with all files details
with url also.
loop the table data and get the content from the files with url by
using curl and convert into base64_encode and indexing.
(by following exact steps from:
http://www.elasticsearch.cn/tutorials/2011/07/18/attachment-type-in-action.html)
please any body provide the solution to decrease the bulk files
indexing time.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4b668549-cd62-439c-91b2-74c1a311fa8f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Binh Ly-2) #2

If you have capacity, you could try creating a few shards first, and then
index in parallel (i.e. from multiple threads).

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6c4d0825-6879-48d0-99f4-efa5460c102f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3