Can you share the code that does the index and get? Get API is full
realtime to the point in time you issued it. What I have seen for example
is users indexing a doc as an async event (on their code), and then doing
realtime get and obviously they have a chance of not "seeing" the latest
index, since its not in ES yet.
We wrap all activity to ES in another service class, but I can share what
our service class calls end up becoming in ES calls.
GET call
Client client = getClient(); // Using TransportClient
GetRequestBuilder requestBuilder = client.prepareGet();
requestBuilder.setIndex(index);
requestBuilder.setType(Elasticsearch.ALL_DOC_TYPES);
requestBuilder.setId(StringUtils.upperCase(id));
ListenableActionFuture resp = requestBuilder.execute();
response = resp.actionGet();
INDEX Call
Client client = getClient(); // Using TransportClient
IndexRequestBuilder doc = client.prepareIndex();
doc.setIndex(index);
doc.setType(Elasticsearch.DEFAULT_DOC_TYPE);
doc.setId(StringUtils.upperCase(id));
doc.setConsistencyLevel(WriteConsistencyLevel.ALL);
doc.setReplicationType(ReplicationType.DEFAULT); // SYNC Default
doc.setSource(fields);
ListenableActionFuture lir = doc.execute();
IndexResponse ir = lir.actionGet();
P.S. Just as a reminder, we end up making a GET, INDEX, and GET again to
validate the real-time values that we changed in the INDEX operation. If
we immediately poll ES for that document, we see the updated version of the
document but not the updated values.
On Wednesday, August 22, 2012 7:45:44 AM UTC-5, kimchy wrote:
Can you share the code that does the index and get? Get API is full
realtime to the point in time you issued it. What I have seen for example
is users indexing a doc as an async event (on their code), and then doing
realtime get and obviously they have a chance of not "seeing" the latest
index, since its not in ES yet.
Regarding quorum, when you index a doc, its sync replication to all
replicas. Write consistency only makes sure there are enough replicas to
index to before the index operation is done.
On Aug 14, 2012, at 9:28 PM, amos.wood <amos...@lifeway.com <javascript:>>
wrote:
We are testing our ES cluster prior to going live and we are having some
issues where the realtime GETs are not in real-time. Using our testing
logic below, we are noticing that after the update is done that the
"version" of the document is coming back always incremented by +1 but the
"old" is being returned instead of the "new" value in ~1% of our tests.
However, if you wait for 1 to 2 more seconds more after step 4 below, the
"new" value will then be returned correctly. We have verified that the
"realtime" setting is set to "true" but we are still having this issue.
Our test involves the following:
- Use the Get API and retrieve a document by an id
- Change a field in the returned document from an "old" to a "new
value.
- Use the Index API and update the document in the index.
- Wait 500 milliseconds after step 3 completes.
- Use the Get API and retrieve the same document again
Cluster Configuration:
- 4 nodes
- 4 shards with 1 replica
- ~40K documents in the index
- Default index refresh of 1 second.
Does anyone have an idea why this is happening?
--
--