Delay before being able to retrieve indexed documents

Elastic Stack Elasticsearch
1

Observation: Using the Java client, after I index a document then I must wait a short time (less than a second) before a wildcard search succeeds. On the other hand, immediately retrieving by document id works. I'm using ElasticSearch 5.5.0 on OS X 10.11.6

Sample code follows:

public class ESJavaClient 
{	
	/** Doc ID for indexing, retrieving and deleting */
	static final String INDEX_1_DOCID = "1";
	
	/** Index name, Index type */
	static final String INDEX_NAME = "my-index", INDEX_TYPE = "my-type";


	void demoBasicIndexing(TransportClient client) {
		String json = "{" +
				"\"user\":\"kimchy\"," +
				"\"postDate\":\"2013-01-30\"," +
				"\"message\":\"trying out Elasticsearch\"" +
				"}";

		IndexResponse indexResponse = client.prepareIndex(INDEX_NAME, INDEX_TYPE, INDEX_1_DOCID)
				.setSource(json, XContentType.JSON)
				.get();
		assert(indexResponse.getResult() == DocWriteResponse.Result.CREATED || 
				indexResponse.getResult() == DocWriteResponse.Result.UPDATED);
	}

	void demoRetrieveAllDocs(TransportClient client) {
		// Without a wait, retrieval fails immediately following index
		System.out.print("Demo retrieve all docs, waiting: ");
		// Keep trying every 50ms
		for (long wait = 0; wait < 5000L; wait+= 50) {
			try {
				Thread.sleep(50L);
				SearchResponse searchResponse = 
						client.prepareSearch(INDEX_NAME)
						.setTypes(INDEX_TYPE)
						.setQuery(QueryBuilders.matchAllQuery()).get();
				SearchHit[] hits = searchResponse.getHits().getHits();
				System.out.print(wait + ",");
				if (hits.length > 0) {
					System.out.println("\nRetrieving: Display results:");
					for (int i = 0; i < hits.length; i++) {
						System.out.println("\t" + hits[i].getSourceAsString());
					}
					break;
				}
			} catch (InterruptedException e) {}
		}		
	}

public static void main( String[] args ) throws UnknownHostException
	{
		// Connect client to cluster
		ESJavaClient esClient = new ESJavaClient();
		
		// The default cluster name is 'elasticsearch' 
		Settings settings = Settings.builder()
				.put("client.transport.sniff", true).build();
		
		TransportClient client =  new PreBuiltTransportClient(settings);
		client.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9300));		

		// Demonstrate basic indexing
		esClient.demoBasicIndexing(client);

		// Retrieve all the entries in the index - this fails if done right away
		esClient.demoRetrieveAllDocs(client);
		
		// Delete a single document - not shown
		esClient.demoDeleteByDocId(client);;

		client.close();
	}
}

Results:
Indexing: Index Response:
IndexResponse[index=my-index,type=my-type,id=1,version=4,result=created,shards={"total":2,"successful":1,"failed":0}]
Demo retrieve all docs, waiting: 0,50,100,150,200,250,300,350,400,
Retrieving: Display results:
{"user":"kimchy","postDate":"2013-01-30","message":"trying out Elasticsearch"}

2

Yes, that is how Elasticsearch works.
Have a look at https://www.elastic.co/guide/en/elasticsearch/guide/2.x/near-real-time.html#refresh-api, it still applies to 5.X.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.