I am planning to ES for indexing(Coming from Lucene) and querying good volume of data. Use case is, 10-20 documents / second(roughly around 40-50 fields) and in parallel doing query.
-- In Lucene i can directly add the document to indexWriter and do the commit in background(Custom thread after n number of minutes). I was using ControlledRealTimeReopenThread and TrackingIndexWriter for query, so that documents are immediately available for search(Even though they are not committed yet). How is it possible to achieve this functionality in ES ? I'll be using REST client for indexing and query.
-- Also do i need to keep track of documents got committed for book-keeping ? What if document was added, but before the commit happened node went down(Assume single node deployment). Now when node comes back, would document be re-indexed automatically(Does ES uses JMS or embedded queues) ? If not, how can i get list of documents got committed ?
Can someone point me to the resource/wiki/example to cover above ?