# Can Elasticsearch Transport Client work without ping **URL:** \ **Category:** Elasticsearch\ **Tags:** es-hadoop\ **Created:** [January 29, 2016, 10:42am UTC](https://discuss.elastic.co/t/can-elasticsearch-transport-client-work-without-ping/40467 "2016-01-29T10:42:06Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![manju.nirmal](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manju.nirmal](https://discuss.elastic.co/u/manju.nirmal)\ **Post date:** [January 29, 2016, 10:42am UTC](https://discuss.elastic.co/t/can-elasticsearch-transport-client-work-without-ping/40467/1 "2016-01-29T10:42:06Z")
Hello all, I have a use case where Elasticsearch Transport Client Java API is used to connect a Hadoop cluster to a remote ES cluster. However, pinging is disabled on the Hadoop cluster as a security feature. Is this the reason why I am getting NoNodeAvailableException while trying to establish connection. Is it required to enable ping or is there any other way? Thanks for your help. Manju ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [January 29, 2016, 10:48am UTC](https://discuss.elastic.co/t/can-elasticsearch-transport-client-work-without-ping/40467/2 "2016-01-29T10:48:40Z")
This forum is mainly about ES-Hadoop project; in your case it looks like you are using custom code. Not sure what you mean by 'pinging' but a quick inspection of the network connectivity and ports between your client and ES should sort things out. ---
**Author:** ![manju.nirmal](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manju.nirmal](https://discuss.elastic.co/u/manju.nirmal)\ **Post date:** [January 29, 2016, 10:55am UTC](https://discuss.elastic.co/t/can-elasticsearch-transport-client-work-without-ping/40467/3 "2016-01-29T10:55:29Z")
I am using the custom node within an ES-Hadoop Project to retrieve index names and types because the index names and types are created dynamically on a daily basis. I went for this solution as dynamic reading is not supported by the Cascading API. ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [January 29, 2016, 11:41am UTC](https://discuss.elastic.co/t/can-elasticsearch-transport-client-work-without-ping/40467/4 "2016-01-29T11:41:46Z")
What do you mean by "dynamic reading"? How does this work in your code? Can provide a quick, high-level function? ---
**Author:** ![manju.nirmal](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manju.nirmal](https://discuss.elastic.co/u/manju.nirmal)\ **Post date:** [January 29, 2016, 1:06pm UTC](https://discuss.elastic.co/t/can-elasticsearch-transport-client-work-without-ping/40467/5 "2016-01-29T13:06:53Z")
I have a function that uses the transport client to connect to ES cluster to retrieve all the existing indices and the types in each indices for specific interval. ``` String[] allIndices = client.admin().cluster().prepareState().execute().actionGet().getState().getMetaData() .concreteAllIndices(); Collections.addAll(listOfIndices, allIndices); Map> indexWithTypes = new HashMap>(); for (String eachIndex : listOfIndices) { List typeNames = new ArrayList(); ImmutableOpenMap indexMapping = client.admin().cluster().prepareState().execute() .actionGet().getState().getMetaData().index(eachIndex).getMappings(); Iterator indexMappingIterator = indexMapping.keysIt(); while (indexMappingIterator.hasNext()) { String mappingsResponseKey = indexMappingIterator.next(); String typeName = indexMapping.get(mappingsResponseKey).type(); typeNames.add(typeName); } indexWithTypes.put(eachIndex, typeNames); return indexWithTypes; ``` Iterate through this using the cascading API to get data from ES to store it in corresponding folders in Hadoop: ``` String indexName = indexWithType.getKey(); List indexTypes = indexWithType.getValue(); for (String indexType : indexTypes) { String hdfsPath = hdfsDir + "/" + indexName + "/" + indexType; ``` The Tap functions are: ``` Tap esInTap = new EsTap(indexName + "/" + indexType, Fields.ALL); Tap hdfsOutTap = new Hfs(new TextLine(new Fields("line")), hdfsPath, SinkMode.UPDATE); ``` ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [January 29, 2016, 6:12pm UTC](https://discuss.elastic.co/t/can-elasticsearch-transport-client-work-without-ping/40467/6 "2016-01-29T18:12:52Z")
Why don't you use an alias instead? You can run a cron job that every X hours/days takes your interval and updates the alias. Then your job would simply point to it - it will be only the alias that would change and your cascading job would remain the same. From your code I can't see how your specific 'interval' is but ES-Hadoop can read from multiple or even all indices. ---
**Author:** ![manju.nirmal](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manju.nirmal](https://discuss.elastic.co/u/manju.nirmal)\ **Post date:** [February 1, 2016, 2:02pm UTC](https://discuss.elastic.co/t/can-elasticsearch-transport-client-work-without-ping/40467/7 "2016-02-01T14:02:58Z")
Found this at the Elasticsearch Hadoop Cascading documentation: ![](https://us1.discourse-cdn.com/elastic/original/2X/1/101f13981623627b908a1143bd48c3b1ff6eff62.png) ![](https://us1.discourse-cdn.com/elastic/original/2X/5/5d605dbfd52a99df46046d857802b9db087617a9.png) My requirement is exactly that, runtime resolution of index names and types. That is why I have implemented the Transport client to retrieve the index names and types and then used them in the cascading workflow to save data into HDFS. I am not sure how alias would help me, because although the index name prefixes remain the same, the types in each index are different. Moreover, I need to store them in correspondingly named folders in HDFS. For example, if index-2015.12.12 has type1 and type2. The folder structure in HDFS should be hfdspath/index-2015.12.12/type1/docs, hfdspath/index-2015.12.12/type2/docs. However if index-2015.12.13 has type1, type2 and type3, then they should be saved under hfdspath/index-2015.12.13/type1/docs, hfdspath/index-2015.12.13/type2/docs and hfdspath/index-2015.12.13/type3/docs ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:26pm UTC](https://discuss.elastic.co/t/can-elasticsearch-transport-client-work-without-ping/40467/8 "2017-07-06T13:26:16Z")