# Using TransportClient within ingest plugin

**URL:** <https://discuss.elastic.co/t/using-transportclient-within-ingest-plugin/78370>\
**Category:** Elasticsearch\
**Created:** [March 13, 2017, 3:21pm UTC](https://discuss.elastic.co/t/using-transportclient-within-ingest-plugin/78370 "2017-03-13T15:21:53Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mkalakota](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mkalakota](https://discuss.elastic.co/u/mkalakota)\
**Post date:** [March 13, 2017, 3:21pm UTC](https://discuss.elastic.co/t/using-transportclient-within-ingest-plugin/78370/1 "2017-03-13T15:21:53Z")

</div>

Hi,

I have two indices.  
One index is populated with data.  
And from other index, when data is injected, I am calling index data (where data is populated) using transport client (via an inject plugin).  
When I try to use this processor (plugin) within pipeline and simulate it, elasticsearch is running into error and exiting.

The error is  
[2017-03-13T20:50:21,673][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [] fatal error in thread [elasticsearch[dSqUz60][management][T#1]], exiting  
java.lang.NoClassDefFoundError: org/apache/http/auth/Credentials  
at org.elasticsearch.index.reindex.ReindexPlugin.getSettings(ReindexPlugin.java:61) ~[?:?]  
at org.elasticsearch.plugins.PluginsService.lambda$getPluginSettings$0(PluginsService.java:83) ~[elasticsearch-5.2.2.jar:5.2.2]  
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267) ~[?:1.8.0\_121]  
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374) ~[?:1.8.0\_121]  
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0\_121]  
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0\_121]  
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0\_121]  
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0\_121]  
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0\_121]  
at org.elasticsearch.plugins.PluginsService.getPluginSettings(PluginsService.java:83) ~[elasticsearch-5.2.2.jar:5.2.2]  
at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:133) ~[elasticsearch-5.2.2.jar:5.2.2]  
at org.elasticsearch.client.transport.TransportClient.(TransportClient.java:258) ~[elasticsearch-5.2.2.jar:5.2.2]  
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:125) ~[?:?]  
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:111) ~[?:?]  
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:101) ~[?:?]  
at com.leoforce.mk.elasticsearch.ingest.LocationCleanser.getClient(LocationCleanser.java:54) ~[?:?]  
at com.leoforce.mk.elasticsearch.ingest.ProfileCleanProcessor.execute(ProfileCleanProcessor.java:78) ~[?:?]  
at org.elasticsearch.ingest.CompoundProcessor.execute(CompoundProcessor.java:100) ~[elasticsearch-5.2.2.jar:5.2.2]  
at org.elasticsearch.ingest.Pipeline.execute(Pipeline.java:58) ~[elasticsearch-5.2.2.jar:5.2.2]  
at org.elasticsearch.action.ingest.SimulateExecutionService.executeDocument(SimulateExecutionService.java:56) ~[elasticsearch-5.2.2.jar:5.2.2]  
at org.elasticsearch.action.ingest.SimulateExecutionService$1.doRun(SimulateExecutionService.java:70) ~[elasticsearch-5.2.2.jar:5.2.2]  
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:596) ~[elasticsearch-5.2.2.jar:5.2.2]  
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.2.2.jar:5.2.2]  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0\_121]  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0\_121]  
at java.lang.Thread.run(Thread.java:745) [?:1.8.0\_121]  
Caused by: java.lang.ClassNotFoundException: org.apache.http.auth.Credentials  
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0\_121]  
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0\_121]  
at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:814) ~[?:1.8.0\_121]  
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0\_121]  
... 26 more

I'm using the client as:  
return new PreBuiltTransportClient(Settings.EMPTY)  
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9300));

Please let me know if I'm missing anything.

Thanks,  
Mouli

---

<div class="post-metadata">

**Author:** ![mkalakota](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mkalakota](https://discuss.elastic.co/u/mkalakota)\
**Post date:** [March 13, 2017, 4:03pm UTC](https://discuss.elastic.co/t/using-transportclient-within-ingest-plugin/78370/2 "2017-03-13T16:03:49Z")

</div>

Adding

 org.apache.httpcomponents httpclient 4.5.3

in plugins maven dependencies fixed this issue.

Not sure if this is the right solution.

Comments are appreciated.

Thanks

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [March 13, 2017, 8:03pm UTC](https://discuss.elastic.co/t/using-transportclient-within-ingest-plugin/78370/3 "2017-03-13T20:03:41Z")

</div>

Hey,

can you add the exact java code here as well? I am not sure if this is a bug, as you should not need that dependency for the TransportClient, but I'd like to be sure.

Also, which Elasticsearch version is this?

--Alex

---

<div class="post-metadata">

**Author:** ![mkalakota](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mkalakota](https://discuss.elastic.co/u/mkalakota)\
**Post date:** [March 14, 2017, 9:03am UTC](https://discuss.elastic.co/t/using-transportclient-within-ingest-plugin/78370/4 "2017-03-14T09:03:40Z")

</div>

Hi Alex,

This in on both elasticsearch version 5.2.0 and 5.2.2  
And the code is,  
public TransportClient getClient() {  
try {  
return new PreBuiltTransportClient(Settings.EMPTY)  
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9300));  
} catch (Exception e) {  
e.printStackTrace();  
}  
return null;  
}

// in another method  
String locationStr = ingestDocument.getFieldValue(FIELD\_LOCATION, String.class, false);  
QueryBuilder builder = QueryBuilders.termQuery("name.keyword", locationStr);  
final TransportClient client = getClient();  
if (null != client) {  
SearchHits hits = client.prepareSearch("locations").setTypes("location").setQuery(builder).get();  
client.close();  
if(hits.totalHits() \> 0){  
Location location = parse(hits);  
ingestDocument.setFieldValue("parsed\_location", location);  
}  
}

And when I debugged, TransportReindexAction.REMOTE\_CLUSTER\_WHITELIST is giving the aforementioned error.

Few other questions I have:

1. How do I ignore a document from my ingest plugin during bulk api? Is there any other way apart from throwing exception?
2. Is creating a client from ingest plugin has any side affects? Is there a way I can close the client when server shutsdown?

Thanks,  
Mouli

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 11, 2017, 9:03am UTC](https://discuss.elastic.co/t/using-transportclient-within-ingest-plugin/78370/5 "2017-04-11T09:03:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.