Are there any examples anywhere displaying the use of the bulk Java API? There's examples via the REST API on the website, but nothing in regards to the Java-side. Basically what I want to do is index a bunch of documents at once, and create the index if it does not already exist.
Currently I have (DataItem is the class which holds a document to index, items is a List, and buildJSON() is a method for building the JSON for a DataItem):
BulkRequestBuilder brb = client.prepareBulk();
for (DataItem di : items)
client.prepareIndex(indexName, typeName, di.get_id())
Which seems to work, if the index already exists. However, this code is essentially me guessing at how the Java bulk API is supposed to be used, as there is no documentation in regards to the Java bulk API on the website. So I have 2 questions:
- (newbie-ish question) Is the above code correct/am I doing it wrong?
- How do I (using the Java API) create the index before handing it off to the bulk API?
- (less newbie-ish) how many documents can/should I throw off to the bulk API at once? As it stands I'm doing them all at once (I'm only prototyping at this point, so collections are small), but how well will it scale? Is thousands ok? Tens of thousands? Hundreds of thousands? Millions?