Index "bootstrapping"

Elastic Stack Elasticsearch
7

Thanks for the information this got me on the right track.

I ended up making two changes:

  • Before making a count query I am using the health api to wait until the
    index comes up:
    client.admin().cluster().prepareHealth(INDEX_NAME).setWaitForYellowStatus().execute().actionGet();

If an IndexMissingException is thrown I know the index needs to be created.

  • When creating the index I had a problem with "put mapping" before the
    index was ready... Instead of using the health api you can also specify a
    mapping when the index is created.

Thanks again,

Ryan

On Tue, Nov 9, 2010 at 7:15 AM, Shay Banon shay.banon@elasticsearch.comwrote:

Make sure to call get or actionGet once you execute an operation,
otherwise, your not blocking on it to return (its executed in an async
manner). Based on the code you posted, you don't call it on index creation.

Regarding know when the index has recovered, you can use the cluster health
API, as was suggested by Lukas.

-shay.banon

On Tue, Nov 9, 2010 at 1:48 AM, Ryan Crumley crumley@gmail.com wrote:

All,

I am integrating Elastic Search into a new java/spring/hibernate/
wicket application. I want to keep this application as standalone as
possible so I opted to integrate ES into the web application itself
(using NodeBuilder.nodeBuilder().data(true).local(false) ). This has
been working great with the exception of "bootstrapping" the index at
startup. The current bootstrap flow looks like this:

  1. Web application is started.
  2. Search service tries to run a count() query to determine if a full
    index is needed.

I run into the first issue here when the index has not completed
loading from the local gateway. This causes the count query fail. If I
wait and retry the count query it eventually completes with the
expected answer and I can decide if I need to reindex the content.

In the case the index does not exist I try to create a new index by
executing a "create index" call:

client.admin().indices().prepareCreate(INDEX_NAME).execute();

Then immediately I want to customize a mapping but sometimes a "put
mapping" call fails because the index is not fully created. Again if I
wait long enough after the "create index" call the "put mapping" will
succeed however it seems like there should be a better way. Any
suggestions?

The data set I am using is around 200-500 small documents, with 3-4
terms. I don't anticipate needing a second ES server any time soon so
ideally any solution would not involve a second ES node. I have tried
using the "local" and "file" gateway with similar results. Since the
current dataset is small if I had to index each time the application
started (no persistence of the index) that would be OK however I still
have the "create index" / "put mapping" race condition.

My current ES configuration is very simple:

gateway.type=local
gateway.recover_after_nodes=0
gateway.recover_after_time=0s

Thanks in advance,

Ryan