# \[Hadoop\] storing data in ES using pig script **URL:** \ **Category:** Elasticsearch\ **Created:** [April 13, 2014, 10:23pm UTC](https://discuss.elastic.co/t/hadoop-storing-data-in-es-using-pig-script/16989 "2014-04-13T22:23:06Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![hanine](https://avatars.discourse-cdn.com/v4/letter/h/b4bc9f/32.png) [@hanine](https://discuss.elastic.co/u/hanine)\ **Post date:** [April 13, 2014, 10:23pm UTC](https://discuss.elastic.co/t/hadoop-storing-data-in-es-using-pig-script/16989/1 "2014-04-13T22:23:06Z")
Hello , I m trying to store data in ES (head) using pig script and it gives me _Input(s):_ _Failed to read data from "/user/hive/warehouse/books"_ _Output(s):_ _Failed to produce result in "books/book"_ I ll be so thankful if someone would like help me -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/979f5688-bd53-4b76-a97a-5b0359c8be75%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/979f5688-bd53-4b76-a97a-5b0359c8be75%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [April 14, 2014, 2:11am UTC](https://discuss.elastic.co/t/hadoop-storing-data-in-es-using-pig-script/16989/2 "2014-04-14T02:11:23Z")
Hi, That isn't a lot of information so it's hard to figure out what's actually work - one can only guess. Can you post your stacktrace/logs and your script pig somewhere - like a gist? One thing that stands out is that you mention you are using Pig yet your path points to a Hive warehouse: > /Failed to read data from "/user/hive/warehouse/books"/ I can infer from this that maybe, the issue, is the fact that you are trying to read a Hive internal file, which Pig can't understand, leading to the error that you see. Cheers, On 4/14/14 1:23 AM, hanine haninne wrote: > Hello , > > I m trying to store data in ES (head) using pig script and it gives me > > /Input(s)😕 > /Failed to read data from "/user/hive/warehouse/books"/ > > /Output(s)😕 > /Failed to produce result in "books/book"/ > > I ll be so thankful if someone would like help me > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/979f5688-bd53-4b76-a97a-5b0359c8be75%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/979f5688-bd53-4b76-a97a-5b0359c8be75%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/979f5688-bd53-4b76-a97a-5b0359c8be75%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/979f5688-bd53-4b76-a97a-5b0359c8be75%40googlegroups.com?utm_medium=email&utm_source=footer). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Costin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/534B43CB.3020507%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/534B43CB.3020507%40gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![hanine](https://avatars.discourse-cdn.com/v4/letter/h/b4bc9f/32.png) [@hanine](https://discuss.elastic.co/u/hanine)\ **Post date:** [April 14, 2014, 8:23am UTC](https://discuss.elastic.co/t/hadoop-storing-data-in-es-using-pig-script/16989/3 "2014-04-14T08:23:14Z")
Hi, Here is my log and my script Pig ## log file : Backend error message java.io.IOException: java.io.IOException: Out of nodes and retries; caught exception at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:469) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:432) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:412) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:256) at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:177) at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: java.io.IOException: Out of nodes and retries; caught exception at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209) at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103) at org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85) at org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60) at org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165) at org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.write(EsOutputFormat.java:147) at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:188) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:586) at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:467) ... 11 more Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at java.net.Socket.(Socket.java:425) at java.net.Socket.(Socket.java:280) at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79) at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121) at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324) at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160) at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74) ... 25 more Pig script: REGISTER /home/hduser/hadoop/lib/elasticsearch-hadoop-1.2.0.jar; weblogs = LOAD '/user/hive/warehouse/weblogs' USING PigStorage('\t') AS (client\_ip : chararray, full\_request\_date : chararray, day : int, month : chararray, month\_num : int, year : int, hour : int, minute : int, second : int, timezone : chararray, http\_verb : chararray, uri : chararray, http\_status\_code : chararray, bytes\_returned : chararray, referrer : chararray, user\_agent : chararray ); weblog\_group = GROUP weblogs by (client\_ip, year, month\_num); weblog\_count = FOREACH weblog\_group GENERATE group.client\_ip, group.year, group.month\_num, COUNT\_STAR(weblogs) as pageviews; STORE weblog\_count INTO 'weblogs2/logs2' USING org.elasticsearch.hadoop.pig.EsStorage(); And what ever I put in the LOAD it gives me the same result,even if I put the path of me desktop Thx Le lundi 14 avril 2014 03:11:23 UTC+1, Costin Leau a écrit : > Hi, > > That isn't a lot of information so it's hard to figure out what's actually > work - one can only guess. Can you post your > stacktrace/logs and your script pig somewhere - like a gist? > > One thing that stands out is that you mention you are using Pig yet your > path points to a Hive warehouse: > > > /Failed to read data from "/user/hive/warehouse/books"/ > > I can infer from this that maybe, the issue, is the fact that you are > trying to read a Hive internal file, which Pig > can't understand, leading to the error that you see. > > Cheers, > > On 4/14/14 1:23 AM, hanine haninne wrote: > > > Hello , > > > > I m trying to store data in ES (head) using pig script and it gives me > > > > /Input(s)😕 > > /Failed to read data from "/user/hive/warehouse/books"/ > > > > /Output(s)😕 > > /Failed to produce result in "books/book"/ > > > > I ll be so thankful if someone would like help me > > > > -- > > You received this message because you are subscribed to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \ \ > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \\>. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/979f5688-bd53-4b76-a97a-5b0359c8be75%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/979f5688-bd53-4b76-a97a-5b0359c8be75%40googlegroups.com) > > > \< > > [https://groups.google.com/d/msgid/elasticsearch/979f5688-bd53-4b76-a97a-5b0359c8be75%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/979f5688-bd53-4b76-a97a-5b0359c8be75%40googlegroups.com?utm_medium=email&utm_source=footer)\>. > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > Costin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [April 14, 2014, 8:33am UTC](https://discuss.elastic.co/t/hadoop-storing-data-in-es-using-pig-script/16989/4 "2014-04-14T08:33:12Z")
Since you are not specifying the network configuration for an elasticsearch node, it will default to localhost:9200. This works as long as you are running Hadoop (Pig, Hive, Cascading, etc...) on the same machine as Elasticsearch - based on your exception that is unlikely the case. Try specifying the `es.nodes` parameter - see the documentation for more information. Additionally, you seem to be using the wrong jar of es-hadoop - in your script you are registering es-hadoop-1.2.0.jar (which does not support the pig/hive/cascading functionality) while the stacktrace indicate you are using es-hadoop-1.3.X.jar. Make sure you are using es-hadoop-1.3.0.M3.jar (which is released and available in Maven Central) and no other version. I recommend starting with the examples in the reference docs, which show to easily load and store data to/from Elasticsearch. Once that works, consider extending your script. Hope this helps, On Mon, Apr 14, 2014 at 11:23 AM, hanine haninne [haninne.5@gmail.com](mailto:haninne.5@gmail.com)wrote: > Hi, > > Here is my log and my script Pig > > ## log file : Backend error message > > java.io.IOException: java.io.IOException: Out of nodes and retries; caught > exception > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:469) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:432) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:412) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:256) > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:177) > at > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.io.IOException: Out of nodes and retries; caught exception > at > org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81) > at > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221) > at > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205) > at > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209) > at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103) > at > org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85) > at > org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60) > at > org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165) > at > org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.write(EsOutputFormat.java:147) > at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:188) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) > at > org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:586) > at > org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:467) > ... 11 more > Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) > at > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > at > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > at > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at java.net.Socket.connect(Socket.java:528) > at java.net.Socket.(Socket.java:425) > at java.net.Socket.(Socket.java:280) > at > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79) > at > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121) > at > org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706) > at > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386) > at > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) > at > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396) > at > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324) > at > org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160) > at > org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74) > ... 25 more > > Pig script: > REGISTER /home/hduser/hadoop/lib/elasticsearch-hadoop-1.2.0.jar; > weblogs = LOAD '/user/hive/warehouse/weblogs' USING PigStorage('\t') > AS (client\_ip : chararray, > full\_request\_date : chararray, > day : int, > month : chararray, > month\_num : int, > year : int, > hour : int, > minute : int, > second : int, > timezone : chararray, > http\_verb : chararray, > uri : chararray, > http\_status\_code : chararray, > bytes\_returned : chararray, > referrer : chararray, > user\_agent : chararray > ); > weblog\_group = GROUP weblogs by (client\_ip, year, month\_num); > weblog\_count = FOREACH weblog\_group GENERATE group.client\_ip, group.year, > group.month\_num, COUNT\_STAR(weblogs) as pageviews; > > STORE weblog\_count INTO 'weblogs2/logs2' USING > org.elasticsearch.hadoop.pig.EsStorage(); > > And what ever I put in the LOAD it gives me the same result,even if I put > the path of me desktop > > Thx > > Le lundi 14 avril 2014 03:11:23 UTC+1, Costin Leau a écrit : > > > Hi, > > > > That isn't a lot of information so it's hard to figure out what's > > actually work - one can only guess. Can you post your > > stacktrace/logs and your script pig somewhere - like a gist? > > > > One thing that stands out is that you mention you are using Pig yet your > > path points to a Hive warehouse: > > > > > /Failed to read data from "/user/hive/warehouse/books"/ > > > > I can infer from this that maybe, the issue, is the fact that you are > > trying to read a Hive internal file, which Pig > > can't understand, leading to the error that you see. > > > > Cheers, > > > > On 4/14/14 1:23 AM, hanine haninne wrote: > > > > > Hello , > > > > > > I m trying to store data in ES (head) using pig script and it gives me > > > > > > /Input(s)😕 > > > /Failed to read data from "/user/hive/warehouse/books"/ > > > > > > /Output(s)😕 > > > /Failed to produce result in "books/book"/ > > > > > > I ll be so thankful if someone would like help me > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \ > > [unsubscribe@googlegroups.com](mailto:unsubscribe@googlegroups.com)\>. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/979f5688-](https://groups.google.com/d/msgid/elasticsearch/979f5688-) > > > bd53-4b76-a97a-5b0359c8be75%[40googlegroups.com](http://40googlegroups.com) > > > \<[https://groups.google.com/d/msgid/elasticsearch/979f5688-](https://groups.google.com/d/msgid/elasticsearch/979f5688-) > > > bd53-4b76-a97a-5b0359c8be75%[40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players](http://40googlegroups.com?utm_medium=) > > > email&utm\_source=footer\>. > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > Costin > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAJogdmdSoOPW9OZ9nf0D4-4m0sV9BCbx7YOH0Z3TVBtGCCi4mw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJogdmdSoOPW9OZ9nf0D4-4m0sV9BCbx7YOH0Z3TVBtGCCi4mw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![hanine](https://avatars.discourse-cdn.com/v4/letter/h/b4bc9f/32.png) [@hanine](https://discuss.elastic.co/u/hanine)\ **Post date:** [April 14, 2014, 1:09pm UTC](https://discuss.elastic.co/t/hadoop-storing-data-in-es-using-pig-script/16989/5 "2014-04-14T13:09:44Z")
Ok ,thank you so much 2014-04-14 9:33 GMT+01:00 Costin Leau [costin.leau@gmail.com](mailto:costin.leau@gmail.com): > Since you are not specifying the network configuration for an > elasticsearch node, it will default to localhost:9200. This works as long > as you are running Hadoop (Pig, Hive, Cascading, etc...) on the same > machine as Elasticsearch - based on your exception that is unlikely the > case. > Try specifying the `es.nodes` parameter - see the documentation for more > information. > > Additionally, you seem to be using the wrong jar of es-hadoop - in your > script you are registering es-hadoop-1.2.0.jar (which does not support the > pig/hive/cascading functionality) while the stacktrace indicate you are > using es-hadoop-1.3.X.jar. > > Make sure you are using es-hadoop-1.3.0.M3.jar (which is released and > available in Maven Central) and no other version. I recommend starting with > the examples in the reference docs, which show to easily load and store > data to/from Elasticsearch. > Once that works, consider extending your script. > > Hope this helps, > > On Mon, Apr 14, 2014 at 11:23 AM, hanine haninne [haninne.5@gmail.com](mailto:haninne.5@gmail.com)wrote: > > > Hi, > > > > Here is my log and my script Pig > > > > ## log file : Backend error message > > > > java.io.IOException: java.io.IOException: Out of nodes and retries; > > caught exception > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:469) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:432) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:412) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:256) > > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:177) > > at > > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649) > > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418) > > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:415) > > at > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > > at org.apache.hadoop.mapred.Child.main(Child.java:249) > > Caused by: java.io.IOException: Out of nodes and retries; caught exception > > at > > org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81) > > at > > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221) > > at > > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205) > > at > > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209) > > at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103) > > at > > org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85) > > at > > org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60) > > at > > org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165) > > at > > org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.write(EsOutputFormat.java:147) > > at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:188) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) > > at > > org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:586) > > at > > org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:467) > > ... 11 more > > Caused by: java.net.ConnectException: Connection refused > > at java.net.PlainSocketImpl.socketConnect(Native Method) > > at > > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > > at > > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > > at > > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > > at java.net.Socket.connect(Socket.java:579) > > at java.net.Socket.connect(Socket.java:528) > > at java.net.Socket.(Socket.java:425) > > at java.net.Socket.(Socket.java:280) > > at > > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79) > > at > > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121) > > at > > org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706) > > at > > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386) > > at > > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) > > at > > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396) > > at > > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324) > > at > > org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160) > > at > > org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74) > > ... 25 more > > > > Pig script: > > REGISTER /home/hduser/hadoop/lib/elasticsearch-hadoop-1.2.0.jar; > > weblogs = LOAD '/user/hive/warehouse/weblogs' USING PigStorage('\t') > > AS (client\_ip : chararray, > > full\_request\_date : chararray, > > day : int, > > month : chararray, > > month\_num : int, > > year : int, > > hour : int, > > minute : int, > > second : int, > > timezone : chararray, > > http\_verb : chararray, > > uri : chararray, > > http\_status\_code : chararray, > > bytes\_returned : chararray, > > referrer : chararray, > > user\_agent : chararray > > ); > > weblog\_group = GROUP weblogs by (client\_ip, year, month\_num); > > weblog\_count = FOREACH weblog\_group GENERATE group.client\_ip, group.year, > > group.month\_num, COUNT\_STAR(weblogs) as pageviews; > > > > STORE weblog\_count INTO 'weblogs2/logs2' USING > > org.elasticsearch.hadoop.pig.EsStorage(); > > > > And what ever I put in the LOAD it gives me the same result,even if I put > > the path of me desktop > > > > Thx > > > > Le lundi 14 avril 2014 03:11:23 UTC+1, Costin Leau a écrit : > > > > > Hi, > > > > > > That isn't a lot of information so it's hard to figure out what's > > > actually work - one can only guess. Can you post your > > > stacktrace/logs and your script pig somewhere - like a gist? > > > > > > One thing that stands out is that you mention you are using Pig yet your > > > path points to a Hive warehouse: > > > > > > > /Failed to read data from "/user/hive/warehouse/books"/ > > > > > > I can infer from this that maybe, the issue, is the fact that you are > > > trying to read a Hive internal file, which Pig > > > can't understand, leading to the error that you see. > > > > > > Cheers, > > > > > > On 4/14/14 1:23 AM, hanine haninne wrote: > > > > > > > Hello , > > > > > > > > I m trying to store data in ES (head) using pig script and it gives me > > > > > > > > /Input(s)😕 > > > > /Failed to read data from "/user/hive/warehouse/books"/ > > > > > > > > /Output(s)😕 > > > > /Failed to produce result in "books/book"/ > > > > > > > > I ll be so thankful if someone would like help me > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \ > > > [unsubscribe@googlegroups.com](mailto:unsubscribe@googlegroups.com)\>. > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/979f5688-](https://groups.google.com/d/msgid/elasticsearch/979f5688-) > > > > bd53-4b76-a97a-5b0359c8be75%[40googlegroups.com](http://40googlegroups.com) > > > > \<[https://groups.google.com/d/msgid/elasticsearch/979f5688-](https://groups.google.com/d/msgid/elasticsearch/979f5688-) > > > > bd53-4b76-a97a-5b0359c8be75%[40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players](http://40googlegroups.com?utm_medium=) > > > > email&utm\_source=footer\>. > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > Costin > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAJogdmdSoOPW9OZ9nf0D4-4m0sV9BCbx7YOH0Z3TVBtGCCi4mw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJogdmdSoOPW9OZ9nf0D4-4m0sV9BCbx7YOH0Z3TVBtGCCi4mw%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAJogdmdSoOPW9OZ9nf0D4-4m0sV9BCbx7YOH0Z3TVBtGCCi4mw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAJogdmdSoOPW9OZ9nf0D4-4m0sV9BCbx7YOH0Z3TVBtGCCi4mw%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CANXJSR\_7p3aUEdJjbU2A3%3DYb%2BW08U4kj%2BEONGh3W7mi30iDqRg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CANXJSR_7p3aUEdJjbU2A3%3DYb%2BW08U4kj%2BEONGh3W7mi30iDqRg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![hanine](https://avatars.discourse-cdn.com/v4/letter/h/b4bc9f/32.png) [@hanine](https://discuss.elastic.co/u/hanine)\ **Post date:** [April 14, 2014, 9:23pm UTC](https://discuss.elastic.co/t/hadoop-storing-data-in-es-using-pig-script/16989/6 "2014-04-14T21:23:52Z")
Hello , I used "elasticsearch-hadoop-1.3.0.M2" and it given me Failed Jobs: JobId Alias Feature Message Outputs job\_201404142111\_0008 weblog\_count,weblog\_group,weblogs GROUP\_BY,COMBINER Message: Job failed! Error - # of failed Reduce Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task\_201404142111\_0008\_r\_000000 weblogs/logs2, Input(s): Failed to read data from "/user/hive/warehouse/weblogs" Output(s): Failed to produce result in "weblogs/logs2" Counters: Total records written : 0 Total bytes written : 0 Spillable Memory Manager spill count : 0 Total bags proactively spilled: 0 Total records proactively spilled: 0 I think it s better to know how things works from the beginning,so pls would u like to tell what I have to do (what should I start with) what should I do to configure elasticsearch (head) with Hadoop and how can I work with elasticsearch head Thank you so much ,really all what u say is so helpful .Thank you 2014-04-14 14:09 GMT+01:00 hanine haninne [haninne.5@gmail.com](mailto:haninne.5@gmail.com): > Ok ,thank you so much > > 2014-04-14 9:33 GMT+01:00 Costin Leau [costin.leau@gmail.com](mailto:costin.leau@gmail.com): > > Since you are not specifying the network configuration for an > > > elasticsearch node, it will default to localhost:9200. This works as long > > as you are running Hadoop (Pig, Hive, Cascading, etc...) on the same > > machine as Elasticsearch - based on your exception that is unlikely the > > case. > > Try specifying the `es.nodes` parameter - see the documentation for more > > information. > > > > Additionally, you seem to be using the wrong jar of es-hadoop - in your > > script you are registering es-hadoop-1.2.0.jar (which does not support the > > pig/hive/cascading functionality) while the stacktrace indicate you are > > using es-hadoop-1.3.X.jar. > > > > Make sure you are using es-hadoop-1.3.0.M3.jar (which is released and > > available in Maven Central) and no other version. I recommend starting with > > the examples in the reference docs, which show to easily load and store > > data to/from Elasticsearch. > > Once that works, consider extending your script. > > > > Hope this helps, > > > > On Mon, Apr 14, 2014 at 11:23 AM, hanine haninne [haninne.5@gmail.com](mailto:haninne.5@gmail.com)wrote: > > > > > Hi, > > > > > > Here is my log and my script Pig > > > > > > ## log file : Backend error message > > > > > > java.io.IOException: java.io.IOException: Out of nodes and retries; > > > caught exception > > > at > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:469) > > > at > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:432) > > > at > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:412) > > > at > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:256) > > > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:177) > > > at > > > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649) > > > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418) > > > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > > > at java.security.AccessController.doPrivileged(Native Method) > > > at javax.security.auth.Subject.doAs(Subject.java:415) > > > at > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > > > at org.apache.hadoop.mapred.Child.main(Child.java:249) > > > Caused by: java.io.IOException: Out of nodes and retries; caught > > > exception > > > at > > > org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81) > > > at > > > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221) > > > at > > > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205) > > > at > > > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209) > > > at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103) > > > at > > > org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85) > > > at > > > org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60) > > > at > > > org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165) > > > at > > > org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.write(EsOutputFormat.java:147) > > > at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:188) > > > at > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) > > > at > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) > > > at > > > org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:586) > > > at > > > org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) > > > at > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:467) > > > ... 11 more > > > Caused by: java.net.ConnectException: Connection refused > > > at java.net.PlainSocketImpl.socketConnect(Native Method) > > > at > > > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > > > at > > > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > > > at > > > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > > > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > > > at java.net.Socket.connect(Socket.java:579) > > > at java.net.Socket.connect(Socket.java:528) > > > at java.net.Socket.(Socket.java:425) > > > at java.net.Socket.(Socket.java:280) > > > at > > > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79) > > > at > > > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121) > > > at > > > org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706) > > > at > > > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386) > > > at > > > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) > > > at > > > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396) > > > at > > > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324) > > > at > > > org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160) > > > at > > > org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74) > > > ... 25 more > > > > > > Pig script: > > > REGISTER /home/hduser/hadoop/lib/elasticsearch-hadoop-1.2.0.jar; > > > weblogs = LOAD '/user/hive/warehouse/weblogs' USING PigStorage('\t') > > > AS (client\_ip : chararray, > > > full\_request\_date : chararray, > > > day : int, > > > month : chararray, > > > month\_num : int, > > > year : int, > > > hour : int, > > > minute : int, > > > second : int, > > > timezone : chararray, > > > http\_verb : chararray, > > > uri : chararray, > > > http\_status\_code : chararray, > > > bytes\_returned : chararray, > > > referrer : chararray, > > > user\_agent : chararray > > > ); > > > weblog\_group = GROUP weblogs by (client\_ip, year, month\_num); > > > weblog\_count = FOREACH weblog\_group GENERATE group.client\_ip, > > > group.year, group.month\_num, COUNT\_STAR(weblogs) as pageviews; > > > > > > STORE weblog\_count INTO 'weblogs2/logs2' USING > > > org.elasticsearch.hadoop.pig.EsStorage(); > > > > > > And what ever I put in the LOAD it gives me the same result,even if I > > > put the path of me desktop > > > > > > Thx > > > > > > Le lundi 14 avril 2014 03:11:23 UTC+1, Costin Leau a écrit : > > > > > > > Hi, > > > > > > > > That isn't a lot of information so it's hard to figure out what's > > > > actually work - one can only guess. Can you post your > > > > stacktrace/logs and your script pig somewhere - like a gist? > > > > > > > > One thing that stands out is that you mention you are using Pig yet > > > > your path points to a Hive warehouse: > > > > > > > > > /Failed to read data from "/user/hive/warehouse/books"/ > > > > > > > > I can infer from this that maybe, the issue, is the fact that you are > > > > trying to read a Hive internal file, which Pig > > > > can't understand, leading to the error that you see. > > > > > > > > Cheers, > > > > > > > > On 4/14/14 1:23 AM, hanine haninne wrote: > > > > > > > > > Hello , > > > > > > > > > > I m trying to store data in ES (head) using pig script and it gives > > > > > me > > > > > > > > > > /Input(s)😕 > > > > > /Failed to read data from "/user/hive/warehouse/books"/ > > > > > > > > > > /Output(s)😕 > > > > > /Failed to produce result in "books/book"/ > > > > > > > > > > I ll be so thankful if someone would like help me > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > send an email to > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \ > > > > [unsubscribe@googlegroups.com](mailto:unsubscribe@googlegroups.com)\>. > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/979f5688-](https://groups.google.com/d/msgid/elasticsearch/979f5688-) > > > > > bd53-4b76-a97a-5b0359c8be75%[40googlegroups.com](http://40googlegroups.com) > > > > > \<[https://groups.google.com/d/msgid/elasticsearch/979f5688-](https://groups.google.com/d/msgid/elasticsearch/979f5688-) > > > > > bd53-4b76-a97a-5b0359c8be75%[40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players](http://40googlegroups.com?utm_medium=) > > > > > email&utm\_source=footer\>. > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > Costin > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAJogdmdSoOPW9OZ9nf0D4-4m0sV9BCbx7YOH0Z3TVBtGCCi4mw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJogdmdSoOPW9OZ9nf0D4-4m0sV9BCbx7YOH0Z3TVBtGCCi4mw%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAJogdmdSoOPW9OZ9nf0D4-4m0sV9BCbx7YOH0Z3TVBtGCCi4mw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAJogdmdSoOPW9OZ9nf0D4-4m0sV9BCbx7YOH0Z3TVBtGCCi4mw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CANXJSR9yKurOp37cr2OLPfF6%2BgE9K7A3DaTi0ecnVWiFJHK96w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CANXJSR9yKurOp37cr2OLPfF6%2BgE9K7A3DaTi0ecnVWiFJHK96w%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![hanine](https://avatars.discourse-cdn.com/v4/letter/h/b4bc9f/32.png) [@hanine](https://discuss.elastic.co/u/hanine)\ **Post date:** [April 15, 2014, 9:10am UTC](https://discuss.elastic.co/t/hadoop-storing-data-in-es-using-pig-script/16989/7 "2014-04-15T09:10:39Z")
Hello , I used "elasticsearch-hadoop-1.3.0.M2" and it given me Failed Jobs: JobId Alias Feature Message Outputs job\_201404142111\_0008 weblog\_count,weblog\_group,weblogs GROUP\_BY,COMBINER Message: Job failed! Error - # of failed Reduce Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task\_201404142111\_0008\_r\_000000 weblogs/logs2, Input(s): Failed to read data from "/user/hive/warehouse/weblogs" Output(s): Failed to produce result in "weblogs/logs2" Counters: Total records written : 0 Total bytes written : 0 Spillable Memory Manager spill count : 0 Total bags proactively spilled: 0 Total records proactively spilled: 0 I think it s better to know how things works from the beginning,so pls would u like to tell what I have to do (what should I start with) what should I do to configure elasticsearch (head) with Hadoop and how can I work with elasticsearch head Thank you so much ,really all what u say is so helpful .Thank you Le lundi 14 avril 2014 09:33:12 UTC+1, Costin Leau a écrit : > Since you are not specifying the network configuration for an > elasticsearch node, it will default to localhost:9200. This works as long > as you are running Hadoop (Pig, Hive, Cascading, etc...) on the same > machine as Elasticsearch - based on your exception that is unlikely the > case. > Try specifying the `es.nodes` parameter - see the documentation for more > information. > > Additionally, you seem to be using the wrong jar of es-hadoop - in your > script you are registering es-hadoop-1.2.0.jar (which does not support the > pig/hive/cascading functionality) while the stacktrace indicate you are > using es-hadoop-1.3.X.jar. > > Make sure you are using es-hadoop-1.3.0.M3.jar (which is released and > available in Maven Central) and no other version. I recommend starting with > the examples in the reference docs, which show to easily load and store > data to/from Elasticsearch. > Once that works, consider extending your script. > > Hope this helps, > > On Mon, Apr 14, 2014 at 11:23 AM, hanine haninne \<[hani...@gmail.com](mailto:hani...@gmail.com)\ > > > wrote: > > > Hi, > > > > Here is my log and my script Pig > > > > ## log file : Backend error message > > > > java.io.IOException: java.io.IOException: Out of nodes and retries; > > caught exception > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:469) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:432) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:412) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:256) > > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:177) > > at > > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649) > > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418) > > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:415) > > at > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > > at org.apache.hadoop.mapred.Child.main(Child.java:249) > > Caused by: java.io.IOException: Out of nodes and retries; caught exception > > at > > org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81) > > at > > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221) > > at > > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205) > > at > > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209) > > at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103) > > at > > org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85) > > at > > org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60) > > at > > org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165) > > at > > org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.write(EsOutputFormat.java:147) > > at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:188) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) > > at > > org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:586) > > at > > org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:467) > > ... 11 more > > Caused by: java.net.ConnectException: Connection refused > > at java.net.PlainSocketImpl.socketConnect(Native Method) > > at > > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > > at > > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > > at > > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > > at java.net.Socket.connect(Socket.java:579) > > at java.net.Socket.connect(Socket.java:528) > > at java.net.Socket.(Socket.java:425) > > at java.net.Socket.(Socket.java:280) > > at > > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79) > > at > > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121) > > at > > org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706) > > at > > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386) > > at > > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) > > at > > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396) > > at > > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324) > > at > > org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160) > > at > > org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74) > > ... 25 more > > > > Pig script: > > REGISTER /home/hduser/hadoop/lib/elasticsearch-hadoop-1.2.0.jar; > > weblogs = LOAD '/user/hive/warehouse/weblogs' USING PigStorage('\t') > > AS (client\_ip : chararray, > > full\_request\_date : chararray, > > day : int, > > month : chararray, > > month\_num : int, > > year : int, > > hour : int, > > minute : int, > > second : int, > > timezone : chararray, > > http\_verb : chararray, > > uri : chararray, > > http\_status\_code : chararray, > > bytes\_returned : chararray, > > referrer : chararray, > > user\_agent : chararray > > ); > > weblog\_group = GROUP weblogs by (client\_ip, year, month\_num); > > weblog\_count = FOREACH weblog\_group GENERATE group.client\_ip, group.year, > > group.month\_num, COUNT\_STAR(weblogs) as pageviews; > > > > STORE weblog\_count INTO 'weblogs2/logs2' USING > > org.elasticsearch.hadoop.pig.EsStorage(); > > > > And what ever I put in the LOAD it gives me the same result,even if I put > > the path of me desktop > > > > Thx > > > > Le lundi 14 avril 2014 03:11:23 UTC+1, Costin Leau a écrit : > > > > > Hi, > > > > > > That isn't a lot of information so it's hard to figure out what's > > > actually work - one can only guess. Can you post your > > > stacktrace/logs and your script pig somewhere - like a gist? > > > > > > One thing that stands out is that you mention you are using Pig yet your > > > path points to a Hive warehouse: > > > > > > > /Failed to read data from "/user/hive/warehouse/books"/ > > > > > > I can infer from this that maybe, the issue, is the fact that you are > > > trying to read a Hive internal file, which Pig > > > can't understand, leading to the error that you see. > > > > > > Cheers, > > > > > > On 4/14/14 1:23 AM, hanine haninne wrote: > > > > > > > Hello , > > > > > > > > I m trying to store data in ES (head) using pig script and it gives me > > > > > > > > /Input(s)😕 > > > > /Failed to read data from "/user/hive/warehouse/books"/ > > > > > > > > /Output(s)😕 > > > > /Failed to produce result in "books/book"/ > > > > > > > > I ll be so thankful if someone would like help me > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \ > > > [unsubscribe@googlegroups.com](mailto:unsubscribe@googlegroups.com)\>. > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/979f5688-](https://groups.google.com/d/msgid/elasticsearch/979f5688-) > > > > bd53-4b76-a97a-5b0359c8be75%[40googlegroups.com](http://40googlegroups.com) > > > > \<[https://groups.google.com/d/msgid/elasticsearch/979f5688-](https://groups.google.com/d/msgid/elasticsearch/979f5688-) > > > > bd53-4b76-a97a-5b0359c8be75%[40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players](http://40googlegroups.com?utm_medium=) > > > > email&utm\_source=footer\>. > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > Costin > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/72350b3d-dc2f-4f5a-b066-cfd0b2241545%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/72350b3d-dc2f-4f5a-b066-cfd0b2241545%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [April 15, 2014, 10:48am UTC](https://discuss.elastic.co/t/hadoop-storing-data-in-es-using-pig-script/16989/8 "2014-04-15T10:48:44Z")
Glad to hear it but know that the latest release is 1.3.0 M3. Simply check the official project page [1] and you get all the info [2], including the download setup from Maven, for both stable and dev/snapshot releases [3] [1] [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/overview/hadoop/) [2] [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/index.html) [3] [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/install.html) On 4/15/14 12:10 PM, hanine haninne wrote: > Hello , > I used "elasticsearch-hadoop-1.3.0.M2" > and it given me > > Failed Jobs: > JobId Alias Feature Message Outputs > job\_201404142111\_0008 weblog\_count,weblog\_group,weblogs GROUP\_BY,COMBINER Message: Job failed! Error - # of > failed Reduce Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task\_201404142111\_0008\_r\_000000 > weblogs/logs2, > > Input(s): > Failed to read data from "/user/hive/warehouse/weblogs" > > Output(s): > Failed to produce result in "weblogs/logs2" > > Counters: > Total records written : 0 > Total bytes written : 0 > Spillable Memory Manager spill count : 0 > Total bags proactively spilled: 0 > Total records proactively spilled: 0 > > I think it s better to know how things works from the beginning,so pls would u like to tell what I have to do (what > should I start with) what should I do to configure elasticsearch (head) with Hadoop and how can I work with > elasticsearch head > > Thank you so much ,really all what u say is so helpful .Thank you > > Le lundi 14 avril 2014 09:33:12 UTC+1, Costin Leau a écrit : > > ``` > Since you are not specifying the network configuration for an elasticsearch node, it will default to localhost:9200. > This works as long as you are running Hadoop (Pig, Hive, Cascading, etc...) on the same machine as Elasticsearch - > based on your exception that is unlikely the case. > Try specifying the `es.nodes` parameter - see the documentation for more information. > > Additionally, you seem to be using the wrong jar of es-hadoop - in your script you are registering > es-hadoop-1.2.0.jar (which does not support the pig/hive/cascading functionality) while the stacktrace indicate you > are using es-hadoop-1.3.X.jar. > > Make sure you are using es-hadoop-1.3.0.M3.jar (which is released and available in Maven Central) and no other > version. I recommend starting with the examples in the reference docs, which show to easily load and store data > to/from Elasticsearch. > Once that works, consider extending your script. > > Hope this helps, > > On Mon, Apr 14, 2014 at 11:23 AM, hanine haninne > wrote: > > Hi, > > Here is my log and my script Pig > > log file : > Backend error message > --------------------- > java.io.IOException: java.io.IOException: Out of nodes and retries; caught exception > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:469) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:432) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:412) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:256) > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:177) > at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.io.IOException: Out of nodes and retries; caught exception > at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81) > at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221) > at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205) > at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209) > at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103) > at org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85) > at org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60) > at org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165) > at org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.write(EsOutputFormat.java:147) > at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:188) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) > at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:586) > at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:467) > ... 11 more > Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at java.net.Socket.connect(Socket.java:528) > at java.net.Socket.(Socket.java:425) > at java.net.Socket.(Socket.java:280) > at > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79) > at > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121) > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706) > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386) > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324) > at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160) > at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74) > ... 25 more > > Pig script: > REGISTER /home/hduser/hadoop/lib/elasticsearch-hadoop-1.2.0.jar; > weblogs = LOAD '/user/hive/warehouse/weblogs' USING PigStorage('\t') > AS (client_ip : chararray, > full_request_date : chararray, > day : int, > month : chararray, > month_num : int, > year : int, > hour : int, > minute : int, > second : int, > timezone : chararray, > http_verb : chararray, > uri : chararray, > http_status_code : chararray, > bytes_returned : chararray, > referrer : chararray, > user_agent : chararray > ); > weblog_group = GROUP weblogs by (client_ip, year, month_num); > weblog_count = FOREACH weblog_group GENERATE group.client_ip, group.year, group.month_num, COUNT_STAR(weblogs) > as pageviews; > > STORE weblog_count INTO 'weblogs2/logs2' USING org.elasticsearch.hadoop.pig.EsStorage(); > > And what ever I put in the LOAD it gives me the same result,even if I put the path of me desktop > > Thx > > Le lundi 14 avril 2014 03:11:23 UTC+1, Costin Leau a écrit : > > Hi, > > That isn't a lot of information so it's hard to figure out what's actually work - one can only guess. Can > you post your > stacktrace/logs and your script pig somewhere - like a gist? > > One thing that stands out is that you mention you are using Pig yet your path points to a Hive warehouse: > > /Failed to read data from "/user/hive/warehouse/books"/ > > I can infer from this that maybe, the issue, is the fact that you are trying to read a Hive internal file, > which Pig > can't understand, leading to the error that you see. > > Cheers, > > On 4/14/14 1:23 AM, hanine haninne wrote: > > Hello , > > > > I m trying to store data in ES (head) using pig script and it gives me > > > > /Input(s):/ > > /Failed to read data from "/user/hive/warehouse/books"/ > > > > /Output(s):/ > > /Failed to produce result in "books/book"/ > > > > I ll be so thankful if someone would like help me > > > > -- > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to > >elasticsearc...@ __googlegroups.com . > > To view this discussion on the web visit > >https://groups.google.com/d/ __msgid/elasticsearch/979f5688-__ bd53-4b76-a97a-5b0359c8be75%__40googlegroups.com > > > >. > > > For more options, visithttps://groups.google.com/d/__optout . > > -- > Costin > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to > elasticsearc...@googlegroups.com . > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/e766ac9d-788e-4614-80ea-04960fdc257f%40googlegroups.com > . > > For more options, visit https://groups.google.com/d/optout . > > ``` > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/72350b3d-dc2f-4f5a-b066-cfd0b2241545%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/72350b3d-dc2f-4f5a-b066-cfd0b2241545%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/72350b3d-dc2f-4f5a-b066-cfd0b2241545%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/72350b3d-dc2f-4f5a-b066-cfd0b2241545%40googlegroups.com?utm_medium=email&utm_source=footer). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Costin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/534D0E8C.3050704%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/534D0E8C.3050704%40gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:35am UTC](https://discuss.elastic.co/t/hadoop-storing-data-in-es-using-pig-script/16989/9 "2017-07-06T01:35:47Z")