Here goes:
public void runESTest() {
final String clusterName = "testcluster";
final String indexName = "testindex";
final String typeName = "testtype";
// delete old data/ directory
deleteDataDirectory();
Node node =
nodeBuilder().clusterName(clusterName).local(true).node();
Client client = node.client();
// create the index
try {
waitForReady(client);
client.admin().indices().create(createIndexRequest(indexName))
.actionGet();
} catch (RemoteTransportException e) {
logger.error("Error creating index {}", indexName, e);
System.exit(1);
}
// index a record
String id = "42";
HashMap<String, Object> data = new HashMap<String, Object>();
data.put("firstname", "Douglas");
data.put("lastname", "Adams");
try {
waitForReady(client);
client.prepareIndex(indexName, typeName, id)
.setSource(buildJSON(data)).execute().actionGet();
waitForRefresh(client, indexName);
} catch (ElasticSearchException e) {
logger.error("Failure to index document with ID: {}", id,
e);
} catch (IOException e) {
logger.error("Failure to create JSON to index", e);
}
// now query for the record.
waitForReady(client);
SearchResponse response = client.prepareSearch(indexName)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(fieldQuery("firstname", "Douglas"))
.addFields("firstname",
"lastname").setFrom(0).setSize(60)
.setExplain(true).execute().actionGet();
System.out.println("Showing hits...");
for (SearchHit sh : response.getHits()) {
System.out.println("Showing hit: '"
+ (String) sh.field("firstname").getValue() +
"'");
}
System.out.println("Done Showing hits...");
// close down client & node
waitForReady(client);
client.close();
node.close();
}
private XContentBuilder buildJSON(HashMap<String, Object> items)
throws IOException {
XContentBuilder xcb = jsonBuilder();
xcb.startObject();
for (String key : items.keySet())
xcb.field(key, items.get(key));
xcb.endObject();
logger.info("JSON for record: {}", xcb.string());
return xcb;
}
void waitForReady(Client client) {
// wait 30 seconds for yellow
ClusterHealthRequest req = new ClusterHealthRequest().timeout(
new TimeValue(30,
TimeUnit.SECONDS)).waitForYellowStatus();
ClusterHealthResponse response =
client.admin().cluster().health(req)
.actionGet();
if (response.getStatus() == ClusterHealthStatus.RED) {
logger.error("Cluster failed to reach ready
status...aborting");
throw new RuntimeException(
"Elasticsearch cluster status failed become
ready");
}
}
private static void waitForRefresh(Client client, String
indexName) {
client.admin().indices().refresh(refreshRequest(indexName));
}
This code prints out the Douglas Adams record when I change the
local() call in the NodeBuilder to be false (at least with v0.14.4),
but prints out no records when it is set to true (in both v0.14.4 and
v0.15.2).
Not sure if I'm refreshing the index correctly, so any feedback on the
waitForRefresh() method would be appreciated.
Thanks.
On Mar 23, 9:37 am, Shay Banon shay.ba...@elasticsearch.com wrote:
A test case would be great. Did you refresh after the indexing is done?
On Wednesday, March 23, 2011 at 6:17 PM, Adam Parkin wrote:
If you mean for doing indexing & queries, then yeah I already use the
prepareXXX methods. I took your response to indicate that creation of
the node/client has an equivalent prepareXXX set of methods, which are
not on the site. I think I just misunderstood what you originally
meant. 
Anywho: I tried incorporating Curtis' suggestions (most of which I was
already doing, the exception being doing a refresh on the index after
indexing). No change, had same problem (indexing completes, but no
matches on query).
I also tried switching to v0.15.2, and ran locally, and had no
change. Indexing completes, but the query itself returns no
matches.
My testing code indexes a single record which consisted of two fields:
firstname & lastname, which had values "Douglas" and "Adams", and the
ID for the record was (of course) set to 42.
The query code looks like
SearchResponse response = client.prepareSearch(INDEX_NAME)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(fieldQuery("firstname", "Douglas"))
.addFields("firstname",
"lastname").setFrom(0).setSize(60)
.setExplain(true).execute().actionGet();
And printing the results of the query looks like:
for (SearchHit sh : response.getHits()) {
System.out.println("Showing hit: '"
- (String) sh.field("firstname").getValue() +
"'");
}
Which prints nothing to the screen (though when I ran the same code
without making the node local, I did get "Douglas" printed on the
screen as expected.
I'll try and write up a simple class which exploits the exact behavior
I'm seeing and post back with it.
Thanks.
On Mar 23, 2:17 am, Shay Banon shay.ba...@elasticsearch.com wrote:
Ohh, and the prepareXXX are the only one that are now documented on the site.
On Wednesday, March 23, 2011 at 11:16 AM, Shay Banon wrote:
If you can gist a simple sample of the code that fails, I can give it a go.
On Wednesday, March 23, 2011 at 6:17 AM, Adam Parkin wrote:
I don't wait for Green, but rather yellow.
And yes, the Client is created as:
Client client = node.client();
As for the prepareXXX calls, I'm not familiar with them (they're not
documented on the site so far as I could see), is there a starting
point I should look at to explore them?
I'll try Curtis' suggested ideas along with trying v0.15+ and report
back tomorrow.
Thanks.
On Mar 22, 2:56 pm, Shay Banon shay.ba...@elasticsearch.com wrote:
Running the local node is aimed exactly at unit tests, and actually, elasticsearch has a lot of tests internally that use it.
Maybe the reason why your code hangs when waiting for a cluster health is that you wait for a GREEN health state, where with a single node and replica count > 0 you will never get.
Dropping the cluster name should not change anything. How do you get the client? Do you use node.client()? (which you should).
On Tuesday, March 22, 2011 at 11:47 PM, Adam Parkin wrote:
I'm trying to setup a unit test for a module I wrote which uses
Elasticsearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.
Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
Elasticsearch Platform — Find real-time answers at scale | Elastic) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.
What I expect to happen is that the node creation code should go from
(in the normal use):
node = nodeBuilder().clusterName(clusterName).node();
to (in the unit test):
node = nodeBuilder().clusterName(clusterName).local(true).node();
And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.
If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.
If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).
Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.
In case it matters, this is running Elasticsearch 0.14.4.
Thanks.