# Issue when indexing to elasticsearch from apache nutch

**URL:** <https://discuss.elastic.co/t/issue-when-indexing-to-elasticsearch-from-apache-nutch/56961>\
**Category:** Elasticsearch\
**Created:** [August 2, 2016, 6:44am UTC](https://discuss.elastic.co/t/issue-when-indexing-to-elasticsearch-from-apache-nutch/56961 "2016-08-02T06:44:23Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sachin\_Shaju](https://avatars.discourse-cdn.com/v4/letter/s/e47c2d/32.png) [@Sachin\_Shaju](https://discuss.elastic.co/u/Sachin_Shaju)\
**Post date:** [August 2, 2016, 6:44am UTC](https://discuss.elastic.co/t/issue-when-indexing-to-elasticsearch-from-apache-nutch/56961/1 "2016-08-02T06:44:23Z")

</div>

I was trying to index from apache nutch to single node ES cluster and got this error.

> org.elasticsearch.transport.RemoteTransportException: Failed to deserialize exception response from stream Caused by: org.elasticsearch.transport.TransportSerializationException: Failed to deserialize exception response from stream at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:173) at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:125) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.StreamCorruptedException: Unsupported version: 1 at org.elasticsearch.common.io.ThrowableObjectInputStream.readStreamHeader(ThrowableObjectInputStream.java:46) at java.io.ObjectInputStream.(ObjectInputStream.java:301) at org.elasticsearch.common.io.ThrowableObjectInputStream.(ThrowableObjectInputStream.java:38) at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:170) ... 23 more

From further research I came to know I should have same jvm version on client and ES server. Reference : [http://jontai.me/blog/2013/06/elasticsearch-remotetransportexception-failed-to-deserialize-exception-response-from-stream/](http://jontai.me/blog/2013/06/elasticsearch-remotetransportexception-failed-to-deserialize-exception-response-from-stream/)

I'm using ES version 2.3.2 and my JVM version is "1.8.0\_91". When I checked /plugins/indexer-elastic/plugin.xml,the version specified is 1.4.1. I would like to know this could be the issue and a possible solution other than to downgrade ES cluster version. I would like to continue with ES 2.3.2. Please help me on this.

PS : The command I've used for indexing is **bin/nutch index crawl/crawldb/ -linkdb crawl/linkdb/ crawl/segments/20160801174223/**

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [August 2, 2016, 7:33am UTC](https://discuss.elastic.co/t/issue-when-indexing-to-elasticsearch-from-apache-nutch/56961/2 "2016-08-02T07:33:28Z")

</div>

This is a question for the Nutch community.

You have to build Nutch from source (master branch). It support ES 2.3.3 [https://github.com/apache/nutch/blob/master/src/plugin/indexer-elastic/plugin.xml](https://github.com/apache/nutch/blob/master/src/plugin/indexer-elastic/plugin.xml)

---

<div class="post-metadata">

**Author:** ![Sachin\_Shaju](https://avatars.discourse-cdn.com/v4/letter/s/e47c2d/32.png) [@Sachin\_Shaju](https://discuss.elastic.co/u/Sachin_Shaju)\
**Post date:** [August 2, 2016, 8:27am UTC](https://discuss.elastic.co/t/issue-when-indexing-to-elasticsearch-from-apache-nutch/56961/3 "2016-08-02T08:27:30Z")

</div>

I asked the question here because the exception was specific to elasticsearch. Thanks for your reply 🙂

---

<div class="post-metadata">

**Author:** ![Sachin\_Shaju](https://avatars.discourse-cdn.com/v4/letter/s/e47c2d/32.png) [@Sachin\_Shaju](https://discuss.elastic.co/u/Sachin_Shaju)\
**Post date:** [August 2, 2016, 12:02pm UTC](https://discuss.elastic.co/t/issue-when-indexing-to-elasticsearch-from-apache-nutch/56961/4 "2016-08-02T12:02:06Z")

</div>

It worked. Thanks @jprante🙂

---

<div class="post-metadata">

**Author:** ![Nestor](https://avatars.discourse-cdn.com/v4/letter/n/e495f1/32.png) [@Nestor](https://discuss.elastic.co/u/Nestor)\
**Post date:** [September 27, 2016, 3:35pm UTC](https://discuss.elastic.co/t/issue-when-indexing-to-elasticsearch-from-apache-nutch/56961/5 "2016-09-27T15:35:00Z")

</div>

Hi Sachin,  
Do you have any info in how you made work? I am trying to hook nutch 1.12 and Elasticsearch 2.4. My website is crawled, I edited the nutch-site.xml. I can see info in port 9200. I just do not know how to see the data. or how to configure fields to display. Any examples?

Thanks,

Nestor

---

<div class="post-metadata">

**Author:** ![Sachin\_Shaju](https://avatars.discourse-cdn.com/v4/letter/s/e47c2d/32.png) [@Sachin\_Shaju](https://discuss.elastic.co/u/Sachin_Shaju)\
**Post date:** [September 28, 2016, 4:16am UTC](https://discuss.elastic.co/t/issue-when-indexing-to-elasticsearch-from-apache-nutch/56961/6 "2016-09-28T04:16:13Z")

</div>

Have you tried crawl script in nutch as _bin/crawl -i urls/ CrawlDir/ 1_ to crawl and index a site ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:16pm UTC](https://discuss.elastic.co/t/issue-when-indexing-to-elasticsearch-from-apache-nutch/56961/7 "2017-07-05T22:16:48Z")

</div>


