Index Routing does not work with Transport Client

Elastic Stack Elasticsearch
1

All,

I am using elastic search version 0.19.4.

My requirement is to index the documents to a particular node and exclude
the rest of the nodes. Hence for multiple index requests. I would manage
which node the index should go to.

I tried the index routing mechanism but this does not seem to work with the
version mentioned above.

My test scenario;
1- started 3 ES nodes with node.tag: node1 , node.tag: node2 and node.tag:
node3 respectively.
2- I am using transport client to communicate the ES nodes.
3- I am using bulk index api to index the document.
4- None of the api available with Transport Client or Bulk request allow me
to hook the index.routing.allocation.include.tag orindex.routing
.allocation.exclude.tag.
5- Used UpdateSettingsRequest but it did not help either.

Sample code;
Settings s =
ImmutableSettings.settingsBuilder().put("testRouting").put("client.transport.sniff",
true).build();
TransportClient tc = new TransportClient(s);
tc.addTransportAddress(new
InetSocketTransportAddress("192.168.1.2", 9300));
tc.addTransportAddress(new
InetSocketTransportAddress("192.168.1.2", 9301));
tc.addTransportAddress(new
InetSocketTransportAddress("192.168.1.2", 9302));

            UpdateSettingsRequest updateSettings = new

UpdateSettingsRequest();
HashMap<String, String> hashMap = new HashMap<String,
String>();
hashMap.put("index.routing.allocation.include.tag","node1"
) ;
hashMap.put("index.routing.allocation.exclude.tag","node2,node3"
) ;
hashMap.put("index.routing.allocation.total_shards_per_node",
"3");
String[] indices = new String[1];
indices[0] ="test*";
updateSettings.indices(indices);
updateSettings.settings(hashMap);

            while (true) { 
                    BulkRequestBuilder bulkRequest = tc.prepareBulk(); 
                    for (int i = 0; i < 3; i++) { 
                            System.out.println( "------" + i); 
                            bulkRequest.add(Requests.indexRequest("test"

  • i).type("type").source("field", "value"));

                      } 
                  bulkRequest.execute().actionGet(); 
          }

Please suggest if I am missing something or the feature is not in 0.19.4.
If not how can I make to work with the available scenario.

Thanks
Amit

I have tried the other option as well to update the settings via
client>admin>indices>updatesettings but even than I am not able to rout;

UpdateSettingsRequest updateSettings = new UpdateSettingsRequest();
HashMap<String, String> hashMap = new HashMap<String,
String>();
hashMap.put("index.routing.allocation.include.tag","node1"
) ;
hashMap.put("index.routing.allocation.exclude.tag","node2,node3"
) ;
hashMap.put("index.routing.allocation.total_shards_per_node",
"3");
updateSettings.settings(hashMap);
String[] indices = new String[3];
indices[0] ="test0";
indices[1] ="test1";
indices[2] ="test2";
updateSettings.indices(indices);

tc.admin().indices().updateSettings(updateSettings).actionGet();

--

2

Hello Shay,

I further explored the ES documents( ES site and google search results) and
the ES google group but I could not find a direct example/sample of
Transport client , Bulk Indexing and index allocation routing combination .

I looked into the test classes and tried other way what ever I can. To
build the above mentioned scenario.

I tried IndexMetaBuilder to build the new index meta and pass it to create
Index api. I also tried Routing api, as follows;

//IndexMetaData metaData = new IndexMetaData();

MetaData metaData = newMetaDataBuilder()

.put(newIndexMetaDataBuilder("test13").settings(settingsBuilder()

.put("index.number_of_shards", 4)

.put("index.number_of_replicas", 0)

//.put("index.routing.allocation.include.tag", "node3")

.put("index.routing.allocation.exclude._ip", "192.168.1.102")

.build()))

.build();

//metaData.

AllocationService strategy = new AllocationService(settingsBuilder()

.build());

RoutingTable routingTable = routingTable()

.add(indexRoutingTable("test13").initializeEmpty(metaData.index("test13"
)))

.build();

ClusterState clusterState = newClusterStateBuilder
().metaData(metaData).routingTable(routingTable).build();

BulkRequestBuilder bulkRequest = tc.prepareBulk();

for (int i = 0; i < 3; i++) {

System.out.println( "------" + i);

IndexRequest source = Requests.indexRequest("test-13" ).type("type"
).source("field", "value");

MappingMetaData mappingMd = null;

if (metaData.hasIndex(source.index())) {

mappingMd = metaData.index(source.index()).mapping(source.type());

}

source.process(metaData, "test-13" , mappingMd, false);

bulkRequest.add(source);

}

bulkRequest.execute().actionGet();
But none of the option seems to work. The index is created in all the nodes
and shards distribute across the all the nodes.

I apologizes if my question is very basic, And I am overlooking or
misunderstood the concept. Please suggest.

Thanks
Amit

On Sunday, September 23, 2012 5:36:39 PM UTC+5:30, Amit Singh wrote:

All,

I am using Elasticsearch version 0.19.4.

My requirement is to index the documents to a particular node and exclude
the rest of the nodes. Hence for multiple index requests. I would manage
which node the index should go to.

I tried the index routing mechanism but this does not seem to work with
the version mentioned above.

My test scenario;
1- started 3 ES nodes with node.tag: node1 , node.tag: node2 and
node.tag: node3 respectively.
2- I am using transport client to communicate the ES nodes.
3- I am using bulk index api to index the document.
4- None of the api available with Transport Client or Bulk request allow
me to hook the index.routing.allocation.include.tag orindex.routing
.allocation.exclude.tag.
5- Used UpdateSettingsRequest but it did not help either.

Sample code;
Settings s =
ImmutableSettings.settingsBuilder().put("testRouting").put("client.transport.sniff",
true).build();
TransportClient tc = new TransportClient(s);
tc.addTransportAddress(new
InetSocketTransportAddress("192.168.1.2", 9300));
tc.addTransportAddress(new
InetSocketTransportAddress("192.168.1.2", 9301));
tc.addTransportAddress(new
InetSocketTransportAddress("192.168.1.2", 9302));

            UpdateSettingsRequest updateSettings = new

UpdateSettingsRequest();
HashMap<String, String> hashMap = new HashMap<String,
String>();
hashMap.put("index.routing.allocation.include.tag","node1"
) ;
hashMap.put("index.routing.allocation.exclude.tag","node2,node3"
) ;
hashMap.put("index.routing.allocation.total_shards_per_node",
"3");
String indices = new String[1];
indices[0] ="test*";
updateSettings.indices(indices);
updateSettings.settings(hashMap);

            while (true) { 
                    BulkRequestBuilder bulkRequest = tc.prepareBulk(); 
                    for (int i = 0; i < 3; i++) { 
                            System.out.println( "------" + i); 
                            bulkRequest.add(Requests.indexRequest("test"

  • i).type("type").source("field", "value"));

                      } 
                  bulkRequest.execute().actionGet(); 
          }

Please suggest if I am missing something or the feature is not in 0.19.4.
If not how can I make to work with the available scenario.

Thanks
Amit

I have tried the other option as well to update the settings via
client>admin>indices>updatesettings but even than I am not able to rout;

UpdateSettingsRequest updateSettings = new UpdateSettingsRequest();
HashMap<String, String> hashMap = new HashMap<String,
String>();
hashMap.put("index.routing.allocation.include.tag","node1"
) ;
hashMap.put("index.routing.allocation.exclude.tag","node2,node3"
) ;
hashMap.put("index.routing.allocation.total_shards_per_node",
"3");
updateSettings.settings(hashMap);
String indices = new String[3];
indices[0] ="test0";
indices[1] ="test1";
indices[2] ="test2";
updateSettings.indices(indices);

tc.admin().indices().updateSettings(updateSettings).actionGet();

--