# Elasticsearch-hadoop 1.3 M3 proxy support not working **URL:** \ **Category:** Elasticsearch\ **Created:** [April 23, 2014, 6:23am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-1-3-m3-proxy-support-not-working/17145 "2014-04-23T06:23:27Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![baskaran\_vaithiyanat](https://avatars.discourse-cdn.com/v4/letter/b/ba9def/32.png) [@baskaran\_vaithiyanat](https://discuss.elastic.co/u/baskaran_vaithiyanat)\ **Post date:** [April 23, 2014, 6:23am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-1-3-m3-proxy-support-not-working/17145/1 "2014-04-23T06:23:27Z")
i am a beginner in elasticsearch and hadoop. i am having a problem with moving data from hdfs into elasticsearch server using es.net.proxy.http.host with credentials. Server secured with credentials using nginx proxy configuration. But when i am trying to move data using pig script it shows null pointer exception. My pig script is _REGISTER elasticsearch-hadoop-1.3.0.M3/dist/elasticsearch-hadoop-1.3.0.M3.jar_ _A = load 'date' using PigStorage() as (date:datetime);_ _store A into 'doc/id' using org.elasticsearch.hadoop.pig.EsStorage('es.net.proxy.http.host=ipaddress','es.net.proxy.http.port=portnumber','es.net.proxy.http.user=username','es.net.proxy.http.pass=password');_ I don't understand where is the problem with my script. Can anyone please help me? Thanks in Advance. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [April 23, 2014, 8:01am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-1-3-m3-proxy-support-not-working/17145/2 "2014-04-23T08:01:32Z")
Hi, Can you post the stacktrace you are getting, I'm not sure what causes the NPE. Can you also confirm the Hadoop/Pig/OS version used? Additionally, could you check your script without the proxy or potentially without authentication - just curious whether it makes any difference. Cheers, On 4/23/14 9:23 AM, baskaran vaithiyanathan wrote: > i am a beginner in elasticsearch and hadoop. i am having a problem with moving data from hdfs into elasticsearch server > using es.net.proxy.http.host with credentials. Server secured with credentials using nginx proxy configuration. But when > i am trying to move data using pig script it shows null pointer exception. > > My pig script is > > _REGISTER elasticsearch-hadoop-1.3.0.M3/dist/elasticsearch-hadoop-1.3.0.M3.jar_ > > _A = load 'date' using PigStorage() as (date:datetime);_ > > _store A into 'doc/id' using > org.elasticsearch.hadoop.pig.EsStorage('es.net.proxy.http.host=ipaddress','es.net.proxy.http.port=portnumber','es.net.proxy.http.user=username','es.net.proxy.http.pass=password');_ > > I don't understand where is the problem with my script. Can anyone please help me? > > Thanks in Advance. > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com?utm_medium=email&utm_source=footer). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Costin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5357735C.1060505%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/5357735C.1060505%40gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![baskaran\_vaithiyanat](https://avatars.discourse-cdn.com/v4/letter/b/ba9def/32.png) [@baskaran\_vaithiyanat](https://discuss.elastic.co/u/baskaran_vaithiyanat)\ **Post date:** [April 23, 2014, 9:10am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-1-3-m3-proxy-support-not-working/17145/3 "2014-04-23T09:10:39Z")
Hi Costin, Thanks for your reply. Please find stackrace and other information that you have mentioned. _1. Stacktrace_ java.lang.NullPointerException at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.setupHttpProxy(CommonsHttpTransport.java:169) at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.(CommonsHttpTransport.java:132) at org.elasticsearch.hadoop.rest.NetworkClient.selectNextNode(NetworkClient.java:67) at org.elasticsearch.hadoop.rest.NetworkClient.(NetworkClient.java:50) at org.elasticsearch.hadoop.rest.RestClient.(RestClient.java:69) at org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:59) at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.init(EsOutputFormat.java:180) at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:157) at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:191) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:638) at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.mapred.Child.main(Child.java:249) _2. I have checked the script without proxy and authentication. it works fine._ _3. Version details_ - ``` OS - Ubuntu 12.04.3 LTS * ``` - ``` ElasticSearch - 1.0.0* ``` - ``` Hadoop - 1.2.1* ``` - ``` ElasticSearch- Hadoop - elasticsearch-hadoop 1.3 M3 * ``` - ``` Pig - 0.12.0* ``` - ``` Nginx - 1.1.19* ``` Thanks. On Wednesday, April 23, 2014 4:01:32 AM UTC-4, Costin Leau wrote: > Hi, > > Can you post the stacktrace you are getting, I'm not sure what causes the > NPE. Can you also confirm the Hadoop/Pig/OS > version used? > Additionally, could you check your script without the proxy or potentially > without authentication - just curious whether > it makes any difference. > > Cheers, > > On 4/23/14 9:23 AM, baskaran vaithiyanathan wrote: > > > i am a beginner in elasticsearch and hadoop. i am having a problem with > > moving data from hdfs into elasticsearch server > > using es.net.proxy.http.host with credentials. Server secured with > > credentials using nginx proxy configuration. But when > > i am trying to move data using pig script it shows null pointer > > exception. > > > > My pig script is > > > > _REGISTER > > elasticsearch-hadoop-1.3.0.M3/dist/elasticsearch-hadoop-1.3.0.M3.jar_ > > > > _A = load 'date' using PigStorage() as (date:datetime);_ > > > > \*store A into 'doc/id' using > > org.elasticsearch.hadoop.pig.EsStorage('es.net.proxy.http.host=ipaddress','es.net.proxy.http.port=portnumber','es.net.proxy.http.user=username','es.net.proxy.http.pass=password');\* > > > I don't understand where is the problem with my script. Can anyone > > please help me? > > > > Thanks in Advance. > > > > -- > > You received this message because you are subscribed to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \ \ > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \\>. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com) > > > \< > > [https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com?utm_medium=email&utm_source=footer)\>. > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > Costin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [April 23, 2014, 9:34am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-1-3-m3-proxy-support-not-working/17145/4 "2014-04-23T09:34:20Z")
Thanks for the detailed report. The user authentication triggered a bug [1] which has been fixed in master - I've already pushed the nightly builds which you can try out [2]. It would be great if you could try it out and report back if it works for you. If you want to stick with M3 until RC1 is released, disabling user authentication should fix the issue. Cheers, [1] [Proxy user authentication causes NPE · Issue #195 · elastic/elasticsearch-hadoop · GitHub](https://github.com/elasticsearch/elasticsearch-hadoop/issues/195) [2] [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/install.html) On 4/23/14 12:10 PM, baskaran vaithiyanathan wrote: > Hi Costin, > > Thanks for your reply. > > Please find stackrace and other information that you have mentioned. > > _1. Stacktrace_ > \* > \* > java.lang.NullPointerException > at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.setupHttpProxy(CommonsHttpTransport.java:169) > at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.(CommonsHttpTransport.java:132) > at org.elasticsearch.hadoop.rest.NetworkClient.selectNextNode(NetworkClient.java:67) > at org.elasticsearch.hadoop.rest.NetworkClient.(NetworkClient.java:50) > at org.elasticsearch.hadoop.rest.RestClient.(RestClient.java:69) > at org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:59) > at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.init(EsOutputFormat.java:180) > at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:157) > at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:191) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) > at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:638) > at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48) > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284) > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277) > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > > _2. I have checked the script without proxy and authentication. it works fine._ > \* > \* > _3. Version details_ > > - > > ``` > OS - Ubuntu 12.04.3 LTS * > > ``` > > - > > ``` > ElasticSearch - 1.0.0* > > ``` > > - > > ``` > Hadoop - 1.2.1* > > ``` > > - > > ``` > ElasticSearch- Hadoop - elasticsearch-hadoop 1.3 M3 * > > ``` > > - > > ``` > Pig - 0.12.0* > > ``` > > - > > ``` > Nginx - 1.1.19*** > > ``` > > Thanks. > > On Wednesday, April 23, 2014 4:01:32 AM UTC-4, Costin Leau wrote: > > ``` > Hi, > > Can you post the stacktrace you are getting, I'm not sure what causes the NPE. Can you also confirm the Hadoop/Pig/OS > version used? > Additionally, could you check your script without the proxy or potentially without authentication - just curious > whether > it makes any difference. > > Cheers, > > On 4/23/14 9:23 AM, baskaran vaithiyanathan wrote: > > i am a beginner in elasticsearch and hadoop. i am having a problem with moving data from hdfs into elasticsearch server > > using es.net.proxy.http.host with credentials. Server secured with credentials using nginx proxy configuration. But when > > i am trying to move data using pig script it shows null pointer exception. > > > > My pig script is > > > > *REGISTER elasticsearch-hadoop-1.3.0.M3/dist/elasticsearch-hadoop-1.3.0.M3.jar* > > > > *A = load 'date' using PigStorage() as (date:datetime);* > > > > *store A into 'doc/id' using > > org.elasticsearch.hadoop.pig.EsStorage('es.net.proxy.http.host=ipaddress','es.net.proxy.http.port=portnumber','es.net.proxy.http.user=username','es.net.proxy.http.pass=password');* > > > > > I don't understand where is the problem with my script. Can anyone please help me? > > > > Thanks in Advance. > > > > -- > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to > >elasticsearc...@googlegroups.com >. > > To view this discussion on the web visit > >https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com > > > >. > > > For more options, visithttps://groups.google.com/d/optout . > > -- > Costin > > ``` > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com?utm_medium=email&utm_source=footer). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Costin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5357891C.4020705%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/5357891C.4020705%40gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![baskaran\_vaithiyanat](https://avatars.discourse-cdn.com/v4/letter/b/ba9def/32.png) [@baskaran\_vaithiyanat](https://discuss.elastic.co/u/baskaran_vaithiyanat)\ **Post date:** [April 23, 2014, 2:34pm UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-1-3-m3-proxy-support-not-working/17145/5 "2014-04-23T14:34:29Z")
Hi Costin, Thanks for the update. I have built a jar file from master. I am getting same NPE in stacktrace even when I try without proxy and authentication. With the new jar i cant push data into Elasticsearch anymore. Please do needful. Thanks. On Wednesday, April 23, 2014 5:34:20 AM UTC-4, Costin Leau wrote: > Thanks for the detailed report. The user authentication triggered a bug > [1] which has been fixed in master - I've > already pushed the nightly builds which you can try out [2]. It would be > great if you could try it out and report back > if it works for you. > > If you want to stick with M3 until RC1 is released, disabling user > authentication should fix the issue. > > Cheers, > > [1] [Proxy user authentication causes NPE · Issue #195 · elastic/elasticsearch-hadoop · GitHub](https://github.com/elasticsearch/elasticsearch-hadoop/issues/195) > [2] > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/install.html) > > On 4/23/14 12:10 PM, baskaran vaithiyanathan wrote: > > > Hi Costin, > > > > Thanks for your reply. > > > > Please find stackrace and other information that you have mentioned. > > > > _1. Stacktrace_ > > \* > > \* > > java.lang.NullPointerException > > at > > org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.setupHttpProxy(CommonsHttpTransport.java:169) > > > at > > org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.(CommonsHttpTransport.java:132) > > > at > > org.elasticsearch.hadoop.rest.NetworkClient.selectNextNode(NetworkClient.java:67) > > > at > > org.elasticsearch.hadoop.rest.NetworkClient.(NetworkClient.java:50) > > at org.elasticsearch.hadoop.rest.RestClient.(RestClient.java:69) > > at > > org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:59) > > > at > > org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.init(EsOutputFormat.java:180) > > > at > > org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:157) > > > at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:191) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) > > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) > > > at > > org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:638) > > > at > > org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) > > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48) > > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284) > > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277) > > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) > > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) > > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:415) > > at > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > > > at org.apache.hadoop.mapred.Child.main(Child.java:249) > > > > _2. I have checked the script without proxy and authentication. it works > > fine._ > > \* > > \* > > _3. Version details_ > > > > - > > > > ``` > > OS - Ubuntu 12.04.3 LTS * > > > > ``` > > > > - > > > > ``` > > ElasticSearch - 1.0.0* > > > > ``` > > > > - > > > > ``` > > Hadoop - 1.2.1* > > > > ``` > > > > - > > > > ``` > > ElasticSearch- Hadoop - elasticsearch-hadoop 1.3 M3 * > > > > ``` > > > > - > > > > ``` > > Pig - 0.12.0* > > > > ``` > > > > - > > > > ``` > > Nginx - 1.1.19*** > > > > ``` > > > > Thanks. > > > > On Wednesday, April 23, 2014 4:01:32 AM UTC-4, Costin Leau wrote: > > > > ``` > > Hi, > > > > Can you post the stacktrace you are getting, I'm not sure what > > > > ``` > > causes the NPE. Can you also confirm the Hadoop/Pig/OS > > > ``` > > version used? > > Additionally, could you check your script without the proxy or > > > > ``` > > potentially without authentication - just curious > > > ``` > > whether > > it makes any difference. > > > > Cheers, > > > > On 4/23/14 9:23 AM, baskaran vaithiyanathan wrote: > > > i am a beginner in elasticsearch and hadoop. i am having a problem > > > > ``` > > with moving data from hdfs into elasticsearch server > > > ``` > > > using es.net.proxy.http.host with credentials. Server secured with > > > > ``` > > credentials using nginx proxy configuration. But when > > > ``` > > > i am trying to move data using pig script it shows null pointer > > > > ``` > > exception. > > > ``` > > > > > > My pig script is > > > > > > *REGISTER > > > > ``` > > elasticsearch-hadoop-1.3.0.M3/dist/elasticsearch-hadoop-1.3.0.M3.jar\* > > > ``` > > > > > > *A = load 'date' using PigStorage() as (date:datetime);* > > > > > > *store A into 'doc/id' using > > > > > > > ``` > > org.elasticsearch.hadoop.pig.EsStorage('es.net.proxy.http.host=ipaddress','es.net.proxy.http.port=portnumber','es.net.proxy.http.user=username','es.net.proxy.http.pass=password');\* > > > ``` > > > > > > I don't understand where is the problem with my script. Can anyone > > > > ``` > > please help me? > > > ``` > > > > > > Thanks in Advance. > > > > > > -- > > > You received this message because you are subscribed to the Google > > > > ``` > > Groups "elasticsearch" group. > > > ``` > > > To unsubscribe from this group and stop receiving emails from it, > > > > ``` > > send an email to > > > ``` > > >elasticsearc...@googlegroups.com > > > ``` > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \ \\>. > > > ``` > > > To view this discussion on the web visit > > > > > > > ``` > > [https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com) > > > ``` > > < > > > > ``` > > [https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com)\> > > > ``` > > > < > > > > ``` > > [https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com?utm_medium=email&utm_source=footer) > > > ``` > > < > > > > ``` > > [https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com?utm_medium=email&utm_source=footer)\>\>. > > > ``` > > > For more options, visithttps://groups.google.com/d/optout < > > > > ``` > > [https://groups.google.com/d/optout](https://groups.google.com/d/optout)\>. > > > ``` > > -- > > Costin > > > > ``` > > > > -- > > You received this message because you are subscribed to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \ \ > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \\>. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com) > > > \< > > [https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com?utm_medium=email&utm_source=footer)\>. > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > Costin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/290b9fb2-4c72-481f-ad44-851cc0669523%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/290b9fb2-4c72-481f-ad44-851cc0669523%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [April 23, 2014, 2:59pm UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-1-3-m3-proxy-support-not-working/17145/6 "2014-04-23T14:59:30Z")
To eliminate any issues/confusion please try the dev snapshots as indicated in the previous email/docs. The latest version in Maven OSS is elasticsearch-hadoop-1.3.0.BUILD-20140423.145248-400.jar This passes the entire test suite just fine and should not give any NPE. If that is not the case, please post the stacktrace to the issue mentioned before, namely [1]. Thanks, [1] [Proxy user authentication causes NPE · Issue #195 · elastic/elasticsearch-hadoop · GitHub](https://github.com/elasticsearch/elasticsearch-hadoop/issues/195) On 4/23/14 5:34 PM, baskaran vaithiyanathan wrote: > Hi Costin, > > Thanks for the update. > > I have built a jar file from master. > > I am getting same NPE in stacktrace even when I try without proxy and authentication. > > With the new jar i cant push data into Elasticsearch anymore. > > Please do needful. > > Thanks. > > On Wednesday, April 23, 2014 5:34:20 AM UTC-4, Costin Leau wrote: > > ``` > Thanks for the detailed report. The user authentication triggered a bug [1] which has been fixed in master - I've > already pushed the nightly builds which you can try out [2]. It would be great if you could try it out and report back > if it works for you. > > If you want to stick with M3 until RC1 is released, disabling user authentication should fix the issue. > > Cheers, > > [1] https://github.com/elasticsearch/elasticsearch-hadoop/issues/195 > > [2] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/install.html > > > On 4/23/14 12:10 PM, baskaran vaithiyanathan wrote: > > Hi Costin, > > > > Thanks for your reply. > > > > Please find stackrace and other information that you have mentioned. > > > > *1. Stacktrace* > > * > > * > > java.lang.NullPointerException > > at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.setupHttpProxy(CommonsHttpTransport.java:169) > > at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.(CommonsHttpTransport.java:132) > > at org.elasticsearch.hadoop.rest.NetworkClient.selectNextNode(NetworkClient.java:67) > > at org.elasticsearch.hadoop.rest.NetworkClient.(NetworkClient.java:50) > > at org.elasticsearch.hadoop.rest.RestClient.(RestClient.java:69) > > at org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:59) > > at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.init(EsOutputFormat.java:180) > > at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:157) > > at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:191) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) > > at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:638) > > at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) > > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48) > > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284) > > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277) > > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) > > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:415) > > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > > at org.apache.hadoop.mapred.Child.main(Child.java:249) > > > > *2. I have checked the script without proxy and authentication. it works fine.* > > * > > * > > *3. Version details* > > * OS - Ubuntu 12.04.3 LTS * > > * ElasticSearch - 1.0.0* > > * Hadoop - 1.2.1* > > * ElasticSearch- Hadoop - elasticsearch-hadoop 1.3 M3 * > > * Pig - 0.12.0* > > * Nginx - 1.1.19*** > > > > Thanks. > > > > On Wednesday, April 23, 2014 4:01:32 AM UTC-4, Costin Leau wrote: > > > > Hi, > > > > Can you post the stacktrace you are getting, I'm not sure what causes the NPE. Can you also confirm the Hadoop/Pig/OS > > version used? > > Additionally, could you check your script without the proxy or potentially without authentication - just curious > > whether > > it makes any difference. > > > > Cheers, > > > > On 4/23/14 9:23 AM, baskaran vaithiyanathan wrote: > > > i am a beginner in elasticsearch and hadoop. i am having a problem with moving data from hdfs into elasticsearch server > > > using es.net.proxy.http.host with credentials. Server secured with credentials using nginx proxy configuration. But when > > > i am trying to move data using pig script it shows null pointer exception. > > > > > > My pig script is > > > > > > *REGISTER elasticsearch-hadoop-1.3.0.M3/dist/elasticsearch-hadoop-1.3.0.M3.jar* > > > > > > *A = load 'date' using PigStorage() as (date:datetime);* > > > > > > *store A into 'doc/id' using > > > org.elasticsearch.hadoop.pig.EsStorage('es.net.proxy.http.host=ipaddress','es.net.proxy.http.port=portnumber','es.net.proxy.http.user=username','es.net.proxy.http.pass=password');* > > > > > > > > > I don't understand where is the problem with my script. Can anyone please help me? > > > > > > Thanks in Advance. > > > > > > -- > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an email to > > >elasticsearc...@googlegroups.com > >. > > > To view this discussion on the web visit > > >https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com > > > > > > > > > > >>. > > > > > > For more options, visithttps://groups.google.com/d/optout >. > > > > -- > > Costin > > > > -- > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to > >elasticsearc...@googlegroups.com >. > > To view this discussion on the web visit > >https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com > > > >. > > > For more options, visithttps://groups.google.com/d/optout . > > -- > Costin > > ``` > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/290b9fb2-4c72-481f-ad44-851cc0669523%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/290b9fb2-4c72-481f-ad44-851cc0669523%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/290b9fb2-4c72-481f-ad44-851cc0669523%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/290b9fb2-4c72-481f-ad44-851cc0669523%40googlegroups.com?utm_medium=email&utm_source=footer). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Costin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5357D552.4080203%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/5357D552.4080203%40gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:34am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-1-3-m3-proxy-support-not-working/17145/7 "2017-07-06T01:34:03Z")