# Web Crawler **URL:** \ **Category:** Elasticsearch\ **Created:** [September 9, 2013, 4:48pm UTC](https://discuss.elastic.co/t/web-crawler/13526 "2013-09-09T16:48:27Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![dhturner14](https://avatars.discourse-cdn.com/v4/letter/d/e8c25b/32.png) [@dhturner14](https://discuss.elastic.co/u/dhturner14)\ **Post date:** [September 9, 2013, 4:48pm UTC](https://discuss.elastic.co/t/web-crawler/13526/1 "2013-09-09T16:48:27Z")
Does anyone actually use Nutch with ElasticSearch? It appears that the integration between the two products does not work. Is there something better to use than Nutch for crawling intranet sites that use ElasticSearch for indexing? Thanks, DH -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dhturner14](https://avatars.discourse-cdn.com/v4/letter/d/e8c25b/32.png) [@dhturner14](https://discuss.elastic.co/u/dhturner14)\ **Post date:** [September 9, 2013, 5:06pm UTC](https://discuss.elastic.co/t/web-crawler/13526/2 "2013-09-09T17:06:22Z")
I failed to mention the error I get... # bin/nutch elasticindex esc1 -all Exception in thread "main" java.lang.RuntimeException: job failed: name=elastic-index [esc1], jobid=job\_local1244330097\_0001 at org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:54) at org.apache.nutch.indexer.elastic.ElasticIndexerJob.run(ElasticIndexerJob.java:52) at org.apache.nutch.indexer.elastic.ElasticIndexerJob.indexElastic(ElasticIndexerJob.java:60) at org.apache.nutch.indexer.elastic.ElasticIndexerJob.run(ElasticIndexerJob.java:73) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.nutch.indexer.elastic.ElasticIndexerJob.main(ElasticIndexerJob.java:78) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\ **Post date:** [September 10, 2013, 7:13am UTC](https://discuss.elastic.co/t/web-crawler/13526/3 "2013-09-10T07:13:16Z")
I don't see any ES specific error. I think this question should go to the Nutch mailing list, have tried to get answers there? On 9 September 2013 19:06, [dhturner14@gmail.com](mailto:dhturner14@gmail.com) wrote: > I failed to mention the error I get... > > # bin/nutch elasticindex esc1 -all > > Exception in thread "main" java.lang.RuntimeException: job failed: > name=elastic-index [esc1], jobid=job\_local1244330097\_0001 > at > org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:54) > at > org.apache.nutch.indexer.elastic.ElasticIndexerJob.run(ElasticIndexerJob.java:52) > at > org.apache.nutch.indexer.elastic.ElasticIndexerJob.indexElastic(ElasticIndexerJob.java:60) > at > org.apache.nutch.indexer.elastic.ElasticIndexerJob.run(ElasticIndexerJob.java:73) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at > org.apache.nutch.indexer.elastic.ElasticIndexerJob.main(ElasticIndexerJob.java:78) > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Met vriendelijke groet, Martijn van Groningen -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [September 10, 2013, 11:07am UTC](https://discuss.elastic.co/t/web-crawler/13526/4 "2013-09-10T11:07:19Z")
Increase the nutch log level so you can see more information. I guess wild, you were runing out of memory, or out of file descriptors. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:17am UTC](https://discuss.elastic.co/t/web-crawler/13526/5 "2017-07-06T02:17:20Z")