Retrieve localhost:9200/_aliases using the java api

Emilie_Lavigne · January 21, 2014, 1:42am

Is there a way to reproduce localhost:9200/_aliases using the java api?

Our system often needs to request the list of indices available (our
indices are organized by date) to identify which days in the query date
range have a corresponding index. We are trying to avoid having to cache
the list since it may change as our content retention policy cleans up
indices.

I can retrieve the indices using the following java api call (equivalent of
curl localhost:9200/_cluster/state):

client.admin().cluster().prepareState().execute().actionGet().getState().metaData()

However on our cluster, this call can take up to 80ms, while the curl get
request for aliases comes back in 15ms. Obviously, we would prefer to use
the latter since the response time is compounded with the search time. I
just can't seem to find a way to get that list using java.

I've tried this, but the ImmutableOpenMap always comes back empty:

ImmutableOpenMap<String, List> aliases =
client.admin().indices().getAliases(new
IndicesGetAliasesRequest()).actionGet().getAliases();

Any suggestions?

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f91edcaf-5ce9-4d92-9aa0-61bd530a21d4%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jprante · January 21, 2014, 7:58pm

Replace "new IndicesGetAliasesRequest()" with "new
GetAliasesRequest(null)", this should work

See also RestGetAliasesAction.java

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHfU2%2B_pto4fngriJ-vn6c_zOs69sayg%2BLe%2BhMNex_gmw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Ivan · January 21, 2014, 7:59pm

The easiest way to find out how to use the Java API equivalent of a REST
call is to simply look up the RestAction class. In this case:

https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/rest/action/admin/indices/alias/get/RestGetIndicesAliasesAction.java https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/rest/action/admin/indices/alias/get/RestGetIndicesAliasesAction.java?source=c

I am on version 0.90.2 and I have been using
client.admin().cluster().state(new
ClusterStateRequest()).actionGet().getState().getMetaData().aliases()
which should be equivalent to yours.

Cheers,

Ivan

On Mon, Jan 20, 2014 at 5:42 PM, Emilie Lavigne emilie.lavigne@gmail.comwrote:

Is there a way to reproduce localhost:9200/_aliases using the java api?

Our system often needs to request the list of indices available (our
indices are organized by date) to identify which days in the query date
range have a corresponding index. We are trying to avoid having to cache
the list since it may change as our content retention policy cleans up
indices.

I can retrieve the indices using the following java api call (equivalent
of curl localhost:9200/_cluster/state):

client.admin().cluster().prepareState().execute().actionGet().getState().metaData()

However on our cluster, this call can take up to 80ms, while the curl get
request for aliases comes back in 15ms. Obviously, we would prefer to use
the latter since the response time is compounded with the search time. I
just can't seem to find a way to get that list using java.

I've tried this, but the ImmutableOpenMap always comes back empty:

ImmutableOpenMap<String, List> aliases =
client.admin().indices().getAliases(new
IndicesGetAliasesRequest()).actionGet().getAliases();

Any suggestions?

Thanks!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f91edcaf-5ce9-4d92-9aa0-61bd530a21d4%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDz0T0RL8qs7arGBrgy4FfiSGvbt2Q6jy2uDhMsixYorQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Emilie_Lavigne · January 21, 2014, 9:19pm

Thanks,

Jôrg's tip didn't work. Null was not accepted in the constructor. We
might be using a different version of Elasticsearch.

However, I did find the 0.90.10 equivalent of the class Ivan suggested.
I'm guessing that's as close as I can get to reproducing the _alias
request. I tried to reproduce it this way (for some reason the action
listener version was not firing for me.. - must have missed something - so
I'm using actionGet() instead):

ClusterStateRequest clusterStateRequest = Requests.clusterStateRequest()
.filterRoutingTable(true)
.filterNodes(true)
.filteredIndices(new String{});

clusterStateRequest.listenerThreaded(false);

client.admin().cluster().state(clusterStateRequest).actionGet().getState().metaData();

Sadly, it was not as fast as what I'm seeing with the curl statement. In
fact, the following call turned out to be faster than the one above and is
what I will be using for now. It's still about 40ms slower than the curl
_alias call, but it's better than the 80ms I was getting with my initial
query.

client.admin().indices().prepareStats().clear().get().getIndices();

On Tuesday, January 21, 2014 2:59:11 PM UTC-5, Ivan Brusic wrote:

The easiest way to find out how to use the Java API equivalent of a REST
call is to simply look up the RestAction class. In this case:

https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/rest/action/admin/indices/alias/get/RestGetIndicesAliasesAction.java https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/rest/action/admin/indices/alias/get/RestGetIndicesAliasesAction.java?source=c

I am on version 0.90.2 and I have been using
client.admin().cluster().state(new
ClusterStateRequest()).actionGet().getState().getMetaData().aliases()
which should be equivalent to yours.

Cheers,

Ivan

On Mon, Jan 20, 2014 at 5:42 PM, Emilie Lavigne <emilie....@gmail.com<javascript:>

wrote:

Is there a way to reproduce localhost:9200/_aliases using the java api?

Our system often needs to request the list of indices available (our
indices are organized by date) to identify which days in the query date
range have a corresponding index. We are trying to avoid having to cache
the list since it may change as our content retention policy cleans up
indices.

I can retrieve the indices using the following java api call (equivalent
of curl localhost:9200/_cluster/state):

client.admin().cluster().prepareState().execute().actionGet().getState().metaData()

However on our cluster, this call can take up to 80ms, while the curl get
request for aliases comes back in 15ms. Obviously, we would prefer to use
the latter since the response time is compounded with the search time. I
just can't seem to find a way to get that list using java.

I've tried this, but the ImmutableOpenMap always comes back empty:

ImmutableOpenMap<String, List> aliases =
client.admin().indices().getAliases(new
IndicesGetAliasesRequest()).actionGet().getAliases();

Any suggestions?

Thanks!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f91edcaf-5ce9-4d92-9aa0-61bd530a21d4%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/566e74a3-8419-42d7-8d16-20ccaecab461%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jprante · January 21, 2014, 10:24pm

Yes, asking the cluster state is possible to filter out for aliases (that
is what the TransportGetAliasesAction is doing, too)

I was using 1.0.0.RC1, RestGetIndicesAliasAction is deprecated in 1.0.0.RC1

A null argument to GetAliasesRequest gives an NPE, but an empty string
works:

    GetAliasesRequest getAliasesRequest = new GetAliasesRequest("");
    GetAliasesResponse response =

client.admin().indices().getAliases(getAliasesRequest).actionGet();
logger.info("{}", response.getAliases());

The curl command can not be faster simply because it calls the Java action
in the background, o I'm wondering what timestamps you are measuring.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGShPx87N8UW2wxzhLBXMLtsV9EvJ5R50oD4S6NaNDEgQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Emilie_Lavigne · January 22, 2014, 8:41pm

Logically, I understand that the curl statement cannot be faster. Which is
how I reckon that I'm doing something wrong. This is how I am timing my
methods (from my own dev machine pointing to a vm) using the Google
stopwatch and ticker.

Client connection to myvm:9300 using java:

ClusterStateRequest clusterStateRequest = Requests.clusterStateRequest()

            .filterRoutingTable(true)
            .filterNodes(true)
            .filteredIndices(new String[]{});

    clusterStateRequest.listenerThreaded(false);

    Stopwatch timer1 = new Stopwatch(Ticker.systemTicker()).start();

searcher.getClient().admin().cluster().state(clusterStateRequest).actionGet().getState().metaData();
System.out.println("myvm:9200/_aliases " +
timer1.stop().elapsedMillis());

=> Gets timed at 60-70ms, while curl -o /dev/null -s -w %{time_total}\n
myvm:9200/_aliases gets timed at 7ms

Client connection to myvm:9300 using java:

Stopwatch timer = new Stopwatch(Ticker.systemTicker()).start();

Map<String, IndexStats> indices =
searcher.getClient().admin().indices().prepareStats().clear().get().getIndices();
System.out.println("myvm:9200/_stats : " + timer.stop().elapsedMillis());

=> gets timed to 25-35ms, while curl -o /dev/null -s -w %{time_total}\n
10.40.1.218:9200/_stats gets timed to 20-30ms

As you can see, the latter gets similar timings in both java and curl,
while the former gets very different timings for what the code inspired by
the class RestGetIndicesAliasesAction.

The other example, I cannot use. I consistently get an empty map back
even if I replace the null with a string.

Emilie

On Tuesday, January 21, 2014 5:24:26 PM UTC-5, Jörg Prante wrote:

Yes, asking the cluster state is possible to filter out for aliases (that
is what the TransportGetAliasesAction is doing, too)

I was using 1.0.0.RC1, RestGetIndicesAliasAction is deprecated in 1.0.0.RC1

A null argument to GetAliasesRequest gives an NPE, but an empty string
works:
    GetAliasesRequest getAliasesRequest = new GetAliasesRequest("");
    GetAliasesResponse response = 
client.admin().indices().getAliases(getAliasesRequest).actionGet();
logger.info("{}", response.getAliases());

The curl command can not be faster simply because it calls the Java action
in the background, o I'm wondering what timestamps you are measuring.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8439c42d-0447-452a-9c5a-c1c737f2260f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Emilie_Lavigne · January 22, 2014, 8:42pm

The ip corresponds to myvm in case you're wondering if that's the cause of
the discrepancy.

On Wednesday, January 22, 2014 3:41:10 PM UTC-5, Emilie Lavigne wrote:

Logically, I understand that the curl statement cannot be faster. Which
is how I reckon that I'm doing something wrong. This is how I am timing my
methods (from my own dev machine pointing to a vm) using the Google
stopwatch and ticker.

Client connection to myvm:9300 using java:

ClusterStateRequest clusterStateRequest = Requests.clusterStateRequest()
            .filterRoutingTable(true)
            .filterNodes(true)
            .filteredIndices(new String[]{});

    clusterStateRequest.listenerThreaded(false);

    Stopwatch timer1 = new Stopwatch(Ticker.systemTicker()).start();
searcher.getClient().admin().cluster().state(clusterStateRequest).actionGet().getState().metaData();
System.out.println("myvm:9200/_aliases " +
timer1.stop().elapsedMillis());

=> Gets timed at 60-70ms, while curl -o /dev/null -s -w %{time_total}\n
myvm:9200/_aliases gets timed at 7ms

Client connection to myvm:9300 using java:

Stopwatch timer = new Stopwatch(Ticker.systemTicker()).start();

Map<String, IndexStats> indices =
searcher.getClient().admin().indices().prepareStats().clear().get().getIndices();
System.out.println("myvm:9200/_stats : " + timer.stop().elapsedMillis());

=> gets timed to 25-35ms, while curl -o /dev/null -s -w %{time_total}\n
10.40.1.218:9200/_stats gets timed to 20-30ms

As you can see, the latter gets similar timings in both java and curl,
while the former gets very different timings for what the code inspired by
the class RestGetIndicesAliasesAction.

The other example, I cannot use. I consistently get an empty map back
even if I replace the null with a string.

Emilie

On Tuesday, January 21, 2014 5:24:26 PM UTC-5, Jörg Prante wrote:
Yes, asking the cluster state is possible to filter out for aliases (that
is what the TransportGetAliasesAction is doing, too)

I was using 1.0.0.RC1, RestGetIndicesAliasAction is deprecated in
1.0.0.RC1

A null argument to GetAliasesRequest gives an NPE, but an empty string
works:
    GetAliasesRequest getAliasesRequest = new GetAliasesRequest("");
    GetAliasesResponse response = 
client.admin().indices().getAliases(getAliasesRequest).actionGet();
logger.info("{}", response.getAliases());

The curl command can not be faster simply because it calls the Java
action in the background, o I'm wondering what timestamps you are measuring.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/566943fa-0356-4d8b-baf3-e5caa47a2be2%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jprante · January 22, 2014, 10:43pm

I have written a test case, in the hope it is useful ... (it's for
1.0.0.RC1)

gist.github.com

https://gist.github.com/jprante/8568886

TestAlias.java


import org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest;
import org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest;
import org.elasticsearch.action.admin.indices.alias.get.GetAliasesResponse;
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.NoNodeAvailableException;
import org.elasticsearch.cluster.block.ClusterBlockException;
import org.elasticsearch.cluster.metadata.AliasAction;
import org.elasticsearch.common.Strings;

This file has been truncated. show original

testalias.log

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running AliasTest
Configuring TestNG with: TestNG652Configurator
[23:39:10,958][INFO ][org.elasticsearch.node   ] [Morlun] version[1.0.0.RC1], pid[27536], build[c6155c5/2014-01-15T17:02:32Z]
[23:39:10,959][INFO ][org.elasticsearch.node   ] [Morlun] initializing ...
[23:39:10,971][INFO ][org.elasticsearch.plugins] [Morlun] loaded [], sites []
[23:39:13,242][INFO ][org.elasticsearch.node   ] [Morlun] initialized
[23:39:13,242][INFO ][org.elasticsearch.node   ] [Morlun] starting ...

This file has been truncated. show original

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHw0zFpdNEhnTjYUgfg%2Bg6kOTt%2BD7y%2BvFELMEX-wuzOnw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.