# ElasticSearch and Hadoop Error

**URL:** <https://discuss.elastic.co/t/elasticsearch-and-hadoop-error/13874>\
**Category:** Elasticsearch\
**Created:** [October 8, 2013, 1:24am UTC](https://discuss.elastic.co/t/elasticsearch-and-hadoop-error/13874 "2013-10-08T01:24:03Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![david\_fauth](https://avatars.discourse-cdn.com/v4/letter/d/e495f1/32.png) [@david\_fauth](https://discuss.elastic.co/u/david_fauth)\
**Post date:** [October 8, 2013, 1:24am UTC](https://discuss.elastic.co/t/elasticsearch-and-hadoop-error/13874/1 "2013-10-08T01:24:03Z")

</div>

I am using MortarData and wanted to try out the new ElasticSearch with Pig.  
The following is the Pig code:

REGISTER '/Users/davidfauth/Downloads/elasticsearch-hadoop-1.3.0.M1.jar';

nucc\_codes = LOAD '/Users/davidfauth/Downloads/nucc\_taxonomy\_130.txt' USING  
PigStorage('\t') AS  
(nuccCode:chararray,  
nuccType:chararray,  
nuccClassification:chararray,  
nuccSpecialty:chararray);

-- ETL the data to your heart content  
B = FOREACH nucc\_codes GENERATE nuccCode, TOTUPLE(nuccType,nuccClassification  
) AS nuccData;  
-- save the result to Elasticsearch  
STORE B INTO '/Users/davidfauth/MortarDataOut/es' USING  
org.elasticsearch.hadoop.pig.ESStorage();

When I run this, it logs the following error:

## ERROR 2013-10-07 21:22:42,398 [main] org.apache.pig.tools.grunt.Grunt: ERROR 2998: Unhandled internal error. org.apache.commons.codec.binary.Base64.encodeBase64([BZZ)[B Pig Stack Trace

ERROR 2998: Unhandled internal error.  
org.apache.commons.codec.binary.Base64.encodeBase64([BZZ)[B

java.lang.NoSuchMethodError:  
org.apache.commons.codec.binary.Base64.encodeBase64([BZZ)[B  
at org.elasticsearch.hadoop.util.IOUtils.serializeToBase64(IOUtils.java:38)  
at org.elasticsearch.hadoop.pig.ESStorage.checkSchema(ESStorage.java:113)  
at  
org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:65)  
at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77)  
at  
org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)  
at  
org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)  
at  
org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)  
at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)  
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)  
at  
org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)  
at  
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:300)  
at org.apache.pig.PigServer.compilePp(PigServer.java:1315)  
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1252)  
at org.apache.pig.PigServer.execute(PigServer.java:1241)  
at org.apache.pig.PigServer.executeBatch(PigServer.java:358)  
at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:134)  
at  
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:195)  
at  
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:167)  
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)  
at org.apache.pig.Main.run(Main.java:433)  
at org.apache.pig.PigRunner.run(PigRunner.java:49)  
at com.mortardata.hawk.HawkMain.runHawkPig(HawkMain.java:26)  
at com.mortardata.hawk.HawkMain.main(HawkMain.java:85)

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\
**Post date:** [October 8, 2013, 7:13am UTC](https://discuss.elastic.co/t/elasticsearch-and-hadoop-error/13874/2 "2013-10-08T07:13:17Z")

</div>

Hi,

I haven't used MortarData with ES-Hadoop. From the looks of it, the  
Hadoop/Pig dependency jars which ES-Hadoop uses internally are not present  
in the classpath. What version of Apache Pig MortarData uses ? can you  
share your classpath?

cheers,

On Tue, Oct 8, 2013 at 4:24 AM, david fauth [dsfauth@gmail.com](mailto:dsfauth@gmail.com) wrote:

> I am using MortarData and wanted to try out the new Elasticsearch with  
> Pig. The following is the Pig code:
> 
> REGISTER '/Users/davidfauth/Downloads/elasticsearch-hadoop-1.3.0.M1.jar';
> 
> nucc\_codes = LOAD '/Users/davidfauth/Downloads/nucc\_taxonomy\_130.txt'  
> USING PigStorage('\t') AS  
> (nuccCode:chararray,  
> nuccType:chararray,  
> nuccClassification:chararray,  
> nuccSpecialty:chararray);
> 
> -- ETL the data to your heart content  
> B = FOREACH nucc\_codes GENERATE nuccCode, TOTUPLE(nuccType,nuccClassification  
> ) AS nuccData;  
> -- save the result to Elasticsearch  
> STORE B INTO '/Users/davidfauth/MortarDataOut/es' USING  
> org.elasticsearch.hadoop.pig.ESStorage();
> 
> When I run this, it logs the following error:
> 
> ## ERROR 2013-10-07 21:22:42,398 [main] org.apache.pig.tools.grunt.Grunt: ERROR 2998: Unhandled internal error. org.apache.commons.codec.binary.Base64.encodeBase64([BZZ)[B Pig Stack Trace
> 
> ERROR 2998: Unhandled internal error.  
> org.apache.commons.codec.binary.Base64.encodeBase64([BZZ)[B
> 
> java.lang.NoSuchMethodError:  
> org.apache.commons.codec.binary.Base64.encodeBase64([BZZ)[B  
> at org.elasticsearch.hadoop.util.IOUtils.serializeToBase64(IOUtils.java:38)  
> at org.elasticsearch.hadoop.pig.ESStorage.checkSchema(ESStorage.java:113)  
> at  
> org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:65)  
> at  
> org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77)  
> at  
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)  
> at  
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)  
> at  
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)  
> at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)  
> at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)  
> at  
> org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)  
> at  
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:300)  
> at org.apache.pig.PigServer.compilePp(PigServer.java:1315)  
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1252)  
> at org.apache.pig.PigServer.execute(PigServer.java:1241)  
> at org.apache.pig.PigServer.executeBatch(PigServer.java:358)  
> at  
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:134)  
> at  
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:195)  
> at  
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:167)  
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)  
> at org.apache.pig.Main.run(Main.java:433)  
> at org.apache.pig.PigRunner.run(PigRunner.java:49)  
> at com.mortardata.hawk.HawkMain.runHawkPig(HawkMain.java:26)  
> at com.mortardata.hawk.HawkMain.main(HawkMain.java:85)
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\
**Post date:** [October 8, 2013, 7:14am UTC](https://discuss.elastic.co/t/elasticsearch-and-hadoop-error/13874/3 "2013-10-08T07:14:27Z")

</div>

To clarify - the jar (commons codec) is present but it seems to be an old  
version.

On Tue, Oct 8, 2013 at 10:13 AM, Costin Leau [costin.leau@gmail.com](mailto:costin.leau@gmail.com) wrote:

> Hi,
> 
> I haven't used MortarData with ES-Hadoop. From the looks of it, the  
> Hadoop/Pig dependency jars which ES-Hadoop uses internally are not present  
> in the classpath. What version of Apache Pig MortarData uses ? can you  
> share your classpath?
> 
> cheers,
> 
> On Tue, Oct 8, 2013 at 4:24 AM, david fauth [dsfauth@gmail.com](mailto:dsfauth@gmail.com) wrote:
> 
> > I am using MortarData and wanted to try out the new Elasticsearch with  
> > Pig. The following is the Pig code:
> > 
> > REGISTER '/Users/davidfauth/Downloads/elasticsearch-hadoop-1.3.0.M1.jar';
> > 
> > nucc\_codes = LOAD '/Users/davidfauth/Downloads/nucc\_taxonomy\_130.txt'  
> > USING PigStorage('\t') AS  
> > (nuccCode:chararray,  
> > nuccType:chararray,  
> > nuccClassification:chararray,  
> > nuccSpecialty:chararray);
> > 
> > -- ETL the data to your heart content  
> > B = FOREACH nucc\_codes GENERATE nuccCode, TOTUPLE(nuccType,nuccClassification  
> > ) AS nuccData;  
> > -- save the result to Elasticsearch  
> > STORE B INTO '/Users/davidfauth/MortarDataOut/es' USING  
> > org.elasticsearch.hadoop.pig.ESStorage();
> > 
> > When I run this, it logs the following error:
> > 
> > ## ERROR 2013-10-07 21:22:42,398 [main] org.apache.pig.tools.grunt.Grunt: ERROR 2998: Unhandled internal error. org.apache.commons.codec.binary.Base64.encodeBase64([BZZ)[B Pig Stack Trace
> > 
> > ERROR 2998: Unhandled internal error.  
> > org.apache.commons.codec.binary.Base64.encodeBase64([BZZ)[B
> > 
> > java.lang.NoSuchMethodError:  
> > org.apache.commons.codec.binary.Base64.encodeBase64([BZZ)[B  
> > at  
> > org.elasticsearch.hadoop.util.IOUtils.serializeToBase64(IOUtils.java:38)  
> > at org.elasticsearch.hadoop.pig.ESStorage.checkSchema(ESStorage.java:113)  
> > at  
> > org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:65)  
> > at  
> > org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77)  
> > at  
> > org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)  
> > at  
> > org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)  
> > at  
> > org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)  
> > at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)  
> > at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)  
> > at  
> > org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)  
> > at  
> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:300)  
> > at org.apache.pig.PigServer.compilePp(PigServer.java:1315)  
> > at  
> > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1252)  
> > at org.apache.pig.PigServer.execute(PigServer.java:1241)  
> > at org.apache.pig.PigServer.executeBatch(PigServer.java:358)  
> > at  
> > org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:134)  
> > at  
> > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:195)  
> > at  
> > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:167)  
> > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)  
> > at org.apache.pig.Main.run(Main.java:433)  
> > at org.apache.pig.PigRunner.run(PigRunner.java:49)  
> > at com.mortardata.hawk.HawkMain.runHawkPig(HawkMain.java:26)  
> > at com.mortardata.hawk.HawkMain.main(HawkMain.java:85)
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:13am UTC](https://discuss.elastic.co/t/elasticsearch-and-hadoop-error/13874/4 "2017-07-06T02:13:23Z")

</div>