[HADOOP] Elasticsearch and hive


(hanine) #1

Hello everybody

I m trying to take data from Hive and put it in elasticsearch
here is the script and the error

hive> ADD JAR
/home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar;
Added /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar to
class path
Added resource:
/home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar
hive> CREATE EXTERNAL TABLE tweetsES (
> id BIGINT,
> created_at STRING,
> source STRING,
> favorited BOOLEAN,
> retweet_count INT,
> retweeted_status STRUCT<
> text:STRING,
> user:STRUCT<screen_name:STRING,name:STRING>>,
> entities STRUCT<
> urls:ARRAY<STRUCT<expanded_url:STRING>>,
> user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
> hashtags:ARRAY<STRUCTtext:STRING>>,
> text STRING,
> user STRUCT<
> screen_name:STRING,
> name:STRING,
> friends_count:INT,
> followers_count:INT,
> statuses_count:INT,
> verified:BOOLEAN,
> utc_offset:INT,
> time_zone:STRING>,
> in_reply_to_screen_name STRING
> )
> STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
> TBLPROPERTIES('es.resource' =
'reseaux_sociaux/tweets','es.input.json' = 'yes');
OK
Time taken: 4.292 seconds
hive> ADD JAR /home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar;
Added /home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar to class path
Added resource: /home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar
hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
java.io.FileNotFoundException: File does not exist:
/home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:558)
at
org.apache.hadoop.filecache.DistributedCache.getFileStatus(DistributedCache.java:185)
at
org.apache.hadoop.filecache.TrackerDistributedCacheManager.getFileStatus(TrackerDistributedCacheManager.java:723)
at
org.apache.hadoop.filecache.TrackerDistributedCacheManager.determineTimestamps(TrackerDistributedCacheManager.java:778)
at
org.apache.hadoop.filecache.TrackerDistributedCacheManager.determineTimestampsAndCacheVisibilities(TrackerDistributedCacheManager.java:755)
at
org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:843)
at
org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:734)
at org.apache.hadoop.mapred.JobClient.access$400(JobClient.java:179)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:951)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
at
org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:447)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:138)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:144)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
Job Submission failed with exception 'java.io.FileNotFoundException(File
does not exist: /home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar)'
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MapRedTask

That path it exist :/home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar
Any help would be so appreciable
Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d1613b48-e58d-4077-a579-92b46a97bcff%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Costin Leau) #2
  1. You don't need to add the hive jars (hive-serdes) to your script - these are already part of the runtime
  2. use es-hadoop-2.0.RC1, 1.3 M2 is fairly old
  3. Make sure that the jar is actually available and that is potentially accessible by Hive. Also do note that
    when running Hive against a cluster, it's best to have the jars available on all nodes hence why most folks copy
    them to HDFS and refer to that location.
    While /home/hduser might be locally available, it might not exist on the Hive server - as indicated by the error.

Hope this helps,

On 5/20/14 3:58 PM, hanine haninne wrote:

Hello everybody

I m trying to take data from Hive and put it in elasticsearch
here is the script and the error

hive> ADD JAR /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar;
Added /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar to class path
Added resource: /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar
hive> CREATE EXTERNAL TABLE tweetsES (
> id BIGINT,
> created_at STRING,
> source STRING,
> favorited BOOLEAN,
> retweet_count INT,
> retweeted_status STRUCT<
> text:STRING,
> user:STRUCT<screen_name:STRING,name:STRING>>,
> entities STRUCT<
> urls:ARRAY<STRUCT<expanded_url:STRING>>,
> user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
> hashtags:ARRAY<STRUCTtext:STRING>>,
> text STRING,
> user STRUCT<
> screen_name:STRING,
> name:STRING,
> friends_count:INT,
> followers_count:INT,
> statuses_count:INT,
> verified:BOOLEAN,
> utc_offset:INT,
> time_zone:STRING>,
> in_reply_to_screen_name STRING
> )
> STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
> TBLPROPERTIES('es.resource' = 'reseaux_sociaux/tweets','es.input.json' = 'yes');
OK
Time taken: 4.292 seconds
hive> ADD JAR /home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar;
Added /home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar to class path
Added resource: /home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar
hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
java.io.FileNotFoundException: File does not exist: /home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:558)
at org.apache.hadoop.filecache.DistributedCache.getFileStatus(DistributedCache.java:185)
at org.apache.hadoop.filecache.TrackerDistributedCacheManager.getFileStatus(TrackerDistributedCacheManager.java:723)
at
org.apache.hadoop.filecache.TrackerDistributedCacheManager.determineTimestamps(TrackerDistributedCacheManager.java:778)
at
org.apache.hadoop.filecache.TrackerDistributedCacheManager.determineTimestampsAndCacheVisibilities(TrackerDistributedCacheManager.java:755)
at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:843)
at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:734)
at org.apache.hadoop.mapred.JobClient.access$400(JobClient.java:179)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:951)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:447)
at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:138)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:144)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
Job Submission failed with exception 'java.io.FileNotFoundException(File does not exist:
/home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask

That path it exist :/home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar
Any help would be so appreciable
Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d1613b48-e58d-4077-a579-92b46a97bcff%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d1613b48-e58d-4077-a579-92b46a97bcff%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/537B549F.8020501%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


(hanine) #3

Thank you so much for ur answer
here is what it gives me now

INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201405210900_0002, Tracking URL =
http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002
Kill Command = /home/hduser/hadoop/libexec/../bin/hadoop job -kill
job_201405210900_0002
Hadoop job information for Stage-0: number of mappers: 1; number of
reducers: 0
2014-05-21 10:12:05,285 Stage-0 map = 0%, reduce = 0%
2014-05-21 10:12:23,408 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201405210900_0002 with errors
Error during job, obtaining debugging information...
Job Tracking URL:
http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002
Examining task ID: task_201405210900_0002_m_000002 (and more) from job
job_201405210900_0002

Task with the most failures(4):

Task ID:
task_201405210900_0002_m_000000

URL:

http://master:50030/taskdetails.jsp?jobid=job_201405210900_0002&tipid=task_201405210900_0002_m_000000

Diagnostic Messages for this Task:
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:426)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
... 22 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassNotFoundException: com.cloudera.hive.serde.JSONSerDe
at
org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:463)
at
org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:479)
at
org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:90)
... 22 more
Caused by: java.lang.ClassNotFoundException:
com.cloudera.hive.serde.JSONSerDe
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
at
org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:422)
... 24 more

FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

2014-05-20 14:11 GMT+01:00 Costin Leau costin.leau@gmail.com:

  1. You don't need to add the hive jars (hive-serdes) to your script -
    these are already part of the runtime
  2. use es-hadoop-2.0.RC1, 1.3 M2 is fairly old
  3. Make sure that the jar is actually available and that is potentially
    accessible by Hive. Also do note that
    when running Hive against a cluster, it's best to have the jars available
    on all nodes hence why most folks copy
    them to HDFS and refer to that location.
    While /home/hduser might be locally available, it might not exist on the
    Hive server - as indicated by the error.

Hope this helps,

On 5/20/14 3:58 PM, hanine haninne wrote:

Hello everybody

I m trying to take data from Hive and put it in elasticsearch
here is the script and the error

hive> ADD JAR /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-
SNAPSHOT.jar;
Added /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar
to class path
Added resource: /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-
SNAPSHOT.jar
hive> CREATE EXTERNAL TABLE tweetsES (
> id BIGINT,
> created_at STRING,
> source STRING,
> favorited BOOLEAN,
> retweet_count INT,
> retweeted_status STRUCT<
> text:STRING,
> user:STRUCT<screen_name:STRING,name:STRING>>,
> entities STRUCT<
> urls:ARRAY<STRUCT<expanded_url:STRING>>,
> user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
> hashtags:ARRAY<STRUCTtext:STRING>>,
> text STRING,
> user STRUCT<
> screen_name:STRING,
> name:STRING,
> friends_count:INT,
> followers_count:INT,
> statuses_count:INT,
> verified:BOOLEAN,
> utc_offset:INT,
> time_zone:STRING>,
> in_reply_to_screen_name STRING
> )
> STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
> TBLPROPERTIES('es.resource' = 'reseaux_sociaux/tweets','es.input.json'
= 'yes');
OK
Time taken: 4.292 seconds
hive> ADD JAR /home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar;
Added /home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar to class path
Added resource: /home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar
hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
java.io.FileNotFoundException: File does not exist: /home/hduser/hadoop/
elasticsearch-hadoop-1.3.0.M2.jar
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(
DistributedFileSystem.java:558)
at org.apache.hadoop.filecache.DistributedCache.getFileStatus(
DistributedCache.java:185)
at org.apache.hadoop.filecache.TrackerDistributedCacheManager
.getFileStatus(TrackerDistributedCacheManager.java:723)
at
org.apache.hadoop.filecache.TrackerDistributedCacheManager
.determineTimestamps(TrackerDistributedCacheManager.java:778)
at
org.apache.hadoop.filecache.TrackerDistributedCacheManager.
determineTimestampsAndCacheVisibilities(TrackerDistributedCacheManager
.java:755)
at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(
JobClient.java:843)
at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(
JobClient.java:734)
at org.apache.hadoop.mapred.JobClient.access$400(JobClient.java:179)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:951)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(
UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(
JobClient.java:936)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(
ExecDriver.java:447)
at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(
MapRedTask.java:138)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:144)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(
TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(
CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(
CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(
CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
Job Submission failed with exception 'java.io.FileNotFoundException(File
does not exist:
/home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.
exec.MapRedTask

That path it exist :/home/hduser/hadoop/elasticsearch-hadoop-1.3.0.M2.jar
Any help would be so appreciable
Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d1613b48-
e58d-4077-a579-92b46a97bcff%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/d1613b48-
e58d-4077-a579-92b46a97bcff%40googlegroups.com?utm_medium=
email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/537B549F.8020501%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Costin Leau) #4

The error indicates that your script refers to a JSonSerDe class (presumably from Cloudera) which cannot be found in the
classpath:

"> Caused by: java.lang.ClassNotFoundException: com.cloudera.hive.serde.JSONSerDe"

In fact if you read through the stacktrace, you'll notice that this comes up while the job configuration
is being read and this class name encountered but the class definition itself cannot be resolved.

I'm not sure what causes this class to be added to the classpath of your script - it could be your script or the client
configuration.
Note the class is not used or added by ES-hadoop itself.

I'm not sure what distro you are using (es-hadoop works across Apache Hadoop, Cloudera, HDP, MapR, Pivotal, etc...) but
you could try running some
basic Hive scripts (without es-hadoop) first to make sure the environment is sane and that you can properly read and
write from HDFS and then add
es-hadoop into the mix.

Hope this helps,

On 5/21/14 12:15 PM, hanine haninne wrote:

Thank you so much for ur answer
here is what it gives me now

INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201405210900_0002, Tracking URL = http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002
Kill Command = /home/hduser/hadoop/libexec/../bin/hadoop job -kill job_201405210900_0002
Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0
2014-05-21 10:12:05,285 Stage-0 map = 0%, reduce = 0%
2014-05-21 10:12:23,408 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201405210900_0002 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002
Examining task ID: task_201405210900_0002_m_000002 (and more) from job job_201405210900_0002

Task with the most failures(4):

Task ID:
task_201405210900_0002_m_000000

URL:
http://master:50030/taskdetails.jsp?jobid=job_201405210900_0002&tipid=task_201405210900_0002_m_000000

Diagnostic Messages for this Task:
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:426)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
... 22 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException:
com.cloudera.hive.serde.JSONSerDe
at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:463)
at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:479)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:90)
... 22 more
Caused by: java.lang.ClassNotFoundException: com.cloudera.hive.serde.JSONSerDe
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:422)
... 24 more

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

2014-05-20 14:11 GMT+01:00 Costin Leau <costin.leau@gmail.com mailto:costin.leau@gmail.com>:

1. You don't need to add the hive jars (hive-serdes) to your script - these are already part of the runtime
2. use es-hadoop-2.0.RC1, 1.3 M2 is fairly old
3. Make sure that the jar is actually available and that is potentially accessible by Hive. Also do note that
when running Hive against a cluster, it's best to have the jars available on all nodes hence why most folks copy
them to HDFS and refer to that location.
While /home/hduser might be locally available, it might not exist on the Hive server - as indicated by the error.

Hope this helps,


On 5/20/14 3:58 PM, hanine haninne wrote:


    Hello everybody

    I m trying to take data from Hive and put it in elasticsearch
    here is the script and the error

    hive> ADD JAR /home/hduser/hadoop/hive-0.11.__0/lib/hive-serdes-1.0-__SNAPSHOT.jar;
    Added /home/hduser/hadoop/hive-0.11.__0/lib/hive-serdes-1.0-__SNAPSHOT.jar to class path
    Added resource: /home/hduser/hadoop/hive-0.11.__0/lib/hive-serdes-1.0-__SNAPSHOT.jar
    hive> CREATE EXTERNAL TABLE tweetsES (
          >    id BIGINT,
          >    created_at STRING,
          >    source STRING,
          >    favorited BOOLEAN,
          >    retweet_count INT,
          >    retweeted_status STRUCT<
          >       text:STRING,
          >       user:STRUCT<screen_name:__STRING,name:STRING>>,
          >    entities STRUCT<
          >       urls:ARRAY<STRUCT<expanded___url:STRING>>,
          >       user_mentions:ARRAY<STRUCT<__screen_name:STRING,name:__STRING>>,
          >       hashtags:ARRAY<STRUCT<text:__STRING>>>,
          >    text STRING,
          >    user STRUCT<
          >       screen_name:STRING,
          >       name:STRING,
          >       friends_count:INT,
          >       followers_count:INT,
          >       statuses_count:INT,
          >       verified:BOOLEAN,
          >       utc_offset:INT,
          >       time_zone:STRING>,
          >    in_reply_to_screen_name STRING
          > )
          > STORED BY 'org.elasticsearch.hadoop.__hive.EsStorageHandler'
          > TBLPROPERTIES('es.resource' = 'reseaux_sociaux/tweets','es.__input.json' = 'yes');
    OK
    Time taken: 4.292 seconds
    hive> ADD JAR /home/hduser/hadoop/__elasticsearch-hadoop-1.3.0.M2.__jar;
    Added /home/hduser/hadoop/__elasticsearch-hadoop-1.3.0.M2.__jar to class path
    Added resource: /home/hduser/hadoop/__elasticsearch-hadoop-1.3.0.M2.__jar
    hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
    Total MapReduce jobs = 1
    Launching Job 1 out of 1
    Number of reduce tasks is set to 0 since there's no reduce operator
    java.io.FileNotFoundException: File does not exist: /home/hduser/hadoop/__elasticsearch-hadoop-1.3.0.M2.__jar
          at org.apache.hadoop.hdfs.__DistributedFileSystem.__getFileStatus(__DistributedFileSystem.java:__558)
          at org.apache.hadoop.filecache.__DistributedCache.__getFileStatus(__DistributedCache.java:185)
          at
    org.apache.hadoop.filecache.__TrackerDistributedCacheManager__.getFileStatus(__TrackerDistributedCacheManager__.java:723)
          at
    org.apache.hadoop.filecache.__TrackerDistributedCacheManager__.determineTimestamps(__TrackerDistributedCacheManager__.java:778)
          at
    org.apache.hadoop.filecache.__TrackerDistributedCacheManager__.__determineTimestampsAndCacheVis__ibilities(__TrackerDistributedCacheManager__.java:755)
          at org.apache.hadoop.mapred.__JobClient.__copyAndConfigureFiles(__JobClient.java:843)
          at org.apache.hadoop.mapred.__JobClient.__copyAndConfigureFiles(__JobClient.java:734)
          at org.apache.hadoop.mapred.__JobClient.access$400(__JobClient.java:179)
          at org.apache.hadoop.mapred.__JobClient$2.run(JobClient.__java:951)
          at org.apache.hadoop.mapred.__JobClient$2.run(JobClient.__java:936)
          at java.security.__AccessController.doPrivileged(__Native Method)
          at javax.security.auth.Subject.__doAs(Subject.java:415)
          at org.apache.hadoop.security.__UserGroupInformation.doAs(__UserGroupInformation.java:__1190)
          at org.apache.hadoop.mapred.__JobClient.submitJobInternal(__JobClient.java:936)
          at org.apache.hadoop.mapred.__JobClient.submitJob(JobClient.__java:910)
          at org.apache.hadoop.hive.ql.__exec.ExecDriver.execute(__ExecDriver.java:447)
          at org.apache.hadoop.hive.ql.__exec.MapRedTask.execute(__MapRedTask.java:138)
          at org.apache.hadoop.hive.ql.__exec.Task.executeTask(Task.__java:144)
          at org.apache.hadoop.hive.ql.__exec.TaskRunner.runSequential(__TaskRunner.java:57)
          at org.apache.hadoop.hive.ql.__Driver.launchTask(Driver.java:__1355)
          at org.apache.hadoop.hive.ql.__Driver.execute(Driver.java:__1139)
          at org.apache.hadoop.hive.ql.__Driver.run(Driver.java:945)
          at org.apache.hadoop.hive.cli.__CliDriver.processLocalCmd(__CliDriver.java:259)
          at org.apache.hadoop.hive.cli.__CliDriver.processCmd(__CliDriver.java:216)
          at org.apache.hadoop.hive.cli.__CliDriver.processLine(__CliDriver.java:413)
          at org.apache.hadoop.hive.cli.__CliDriver.run(CliDriver.java:__756)
          at org.apache.hadoop.hive.cli.__CliDriver.main(CliDriver.java:__614)
          at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native Method)
          at sun.reflect.__NativeMethodAccessorImpl.__invoke(__NativeMethodAccessorImpl.java:__57)
          at sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__DelegatingMethodAccessorImpl.__java:43)
          at java.lang.reflect.Method.__invoke(Method.java:606)
          at org.apache.hadoop.util.RunJar.__main(RunJar.java:160)
    Job Submission failed with exception 'java.io <http://java.io>.__FileNotFoundException(File does not exist:
    /home/hduser/hadoop/__elasticsearch-hadoop-1.3.0.M2.__jar)'
    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.__exec.MapRedTask


    That path it exist :/home/hduser/hadoop/__elasticsearch-hadoop-1.3.0.M2.__jar
    Any help would be so appreciable
    Thanks

    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>>.

    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/d1613b48-__e58d-4077-a579-92b46a97bcff%__40googlegroups.com
    <https://groups.google.com/d/msgid/elasticsearch/d1613b48-e58d-4077-a579-92b46a97bcff%40googlegroups.com>
    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-__e58d-4077-a579-92b46a97bcff%__40googlegroups.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/d1613b48-e58d-4077-a579-92b46a97bcff%40googlegroups.com?utm_medium=email&utm_source=footer>>.

    For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.


--
Costin


--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/537B549F.__8020501%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/537B549F.8020501%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%40mail.gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/537C738E.9030001%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


(hanine) #5

yes, you are right.Thanks so much

well I tried hive with HDFS and it works very well,by the way I m using
hive-0.11.0 , hadoop-1.2.1 and elasticsearch-1.0.1
Here is the script of the tables
CREATE EXTERNAL TABLE tweets (
id BIGINT,
created_at STRING,
source STRING,
favorited BOOLEAN,
retweet_count INT,
retweeted_status STRUCT<
text:STRING,
user:STRUCT<screen_name:STRING,name:STRING>>,
entities STRUCT<
urls:ARRAY<STRUCT<expanded_url:STRING>>,
user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
hashtags:ARRAY<STRUCTtext:STRING>>,
text STRING,
user STRUCT<
screen_name:STRING,
name:STRING,
friends_count:INT,
followers_count:INT,
statuses_count:INT,
verified:BOOLEAN,
utc_offset:INT,
time_zone:STRING>,
in_reply_to_screen_name STRING
)
ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe'
LOCATION '/flume/Tweets_A';

do I have to add "com.cloudera.hive.serde.JSONSerDe'" an other time to put
data in elasticsearch ?? if yes,How !!

Thank you so very much

2014-05-21 10:36 GMT+01:00 Costin Leau costin.leau@gmail.com:

The error indicates that your script refers to a JSonSerDe class
(presumably from Cloudera) which cannot be found in the classpath:

"> Caused by: java.lang.ClassNotFoundException: com.cloudera.hive.serde.
JSONSerDe"

In fact if you read through the stacktrace, you'll notice that this comes
up while the job configuration
is being read and this class name encountered but the class definition
itself cannot be resolved.

I'm not sure what causes this class to be added to the classpath of your
script - it could be your script or the client configuration.
Note the class is not used or added by ES-hadoop itself.

I'm not sure what distro you are using (es-hadoop works across Apache
Hadoop, Cloudera, HDP, MapR, Pivotal, etc...) but you could try running some
basic Hive scripts (without es-hadoop) first to make sure the environment
is sane and that you can properly read and write from HDFS and then add
es-hadoop into the mix.

Hope this helps,

On 5/21/14 12:15 PM, hanine haninne wrote:

Thank you so much for ur answer
here is what it gives me now

INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;

Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201405210900_0002, Tracking URL = http://master:50030/
jobdetails.jsp?jobid=job_201405210900_0002
Kill Command = /home/hduser/hadoop/libexec/../bin/hadoop job -kill
job_201405210900_0002
Hadoop job information for Stage-0: number of mappers: 1; number of
reducers: 0
2014-05-21 10:12:05,285 Stage-0 map = 0%, reduce = 0%
2014-05-21 10:12:23,408 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201405210900_0002 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://master:50030/jobdetails.jsp?jobid=job_
201405210900_0002
Examining task ID: task_201405210900_0002_m_000002 (and more) from job
job_201405210900_0002

Task with the most failures(4):

Task ID:
task_201405210900_0002_m_000000

URL:
http://master:50030/taskdetails.jsp?jobid=job_
201405210900_0002&tipid=task_201405210900_0002_m_000000

Diagnostic Messages for this Task:
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(
ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(
ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(
ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:426)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(
UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(
ReflectionUtils.java:88)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(
ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(
ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(
ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(
ReflectionUtils.java:88)
... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(
ExecMapper.java:121)
... 22 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.
ClassNotFoundException:
com.cloudera.hive.serde.JSONSerDe
at org.apache.hadoop.hive.ql.exec.MapOperator.
getConvertedOI(MapOperator.java:463)
at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(
MapOperator.java:479)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(
ExecMapper.java:90)
... 22 more
Caused by: java.lang.ClassNotFoundException: com.cloudera.hive.serde.
JSONSerDe
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at org.apache.hadoop.conf.Configuration.getClassByName(
Configuration.java:810)
at org.apache.hadoop.hive.ql.exec.MapOperator.
getConvertedOI(MapOperator.java:422)
... 24 more

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.
exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

2014-05-20 14:11 GMT+01:00 Costin Leau <costin.leau@gmail.com <mailto:
costin.leau@gmail.com>>:

1. You don't need to add the hive jars (hive-serdes) to your script -

these are already part of the runtime
2. use es-hadoop-2.0.RC1, 1.3 M2 is fairly old
3. Make sure that the jar is actually available and that is
potentially accessible by Hive. Also do note that
when running Hive against a cluster, it's best to have the jars
available on all nodes hence why most folks copy
them to HDFS and refer to that location.
While /home/hduser might be locally available, it might not exist on
the Hive server - as indicated by the error.

Hope this helps,


On 5/20/14 3:58 PM, hanine haninne wrote:


    Hello everybody

    I m trying to take data from Hive and put it in elasticsearch
    here is the script and the error

    hive> ADD JAR /home/hduser/hadoop/hive-0.11.

__0/lib/hive-serdes-1.0-__SNAPSHOT.jar;
Added /home/hduser/hadoop/hive-0.11.__0/lib/hive-serdes-1.0-__SNAPSHOT.jar
to class path
Added resource: /home/hduser/hadoop/hive-0.11.
__0/lib/hive-serdes-1.0-__SNAPSHOT.jar

    hive> CREATE EXTERNAL TABLE tweetsES (
          >    id BIGINT,
          >    created_at STRING,
          >    source STRING,
          >    favorited BOOLEAN,
          >    retweet_count INT,
          >    retweeted_status STRUCT<
          >       text:STRING,
          >       user:STRUCT<screen_name:__STRING,name:STRING>>,
          >    entities STRUCT<
          >       urls:ARRAY<STRUCT<expanded___url:STRING>>,
          >       user_mentions:ARRAY<STRUCT<__

screen_name:STRING,name:__STRING>>,
> hashtags:ARRAY<STRUCTtext:__STRING>>,

          >    text STRING,
          >    user STRUCT<
          >       screen_name:STRING,
          >       name:STRING,
          >       friends_count:INT,
          >       followers_count:INT,
          >       statuses_count:INT,
          >       verified:BOOLEAN,
          >       utc_offset:INT,
          >       time_zone:STRING>,
          >    in_reply_to_screen_name STRING
          > )
          > STORED BY 'org.elasticsearch.hadoop.__

hive.EsStorageHandler'
> TBLPROPERTIES('es.resource' =
'reseaux_sociaux/tweets','es.__input.json' = 'yes');

    OK
    Time taken: 4.292 seconds
    hive> ADD JAR /home/hduser/hadoop/__

elasticsearch-hadoop-1.3.0.M2.__jar;
Added /home/hduser/hadoop/__elasticsearch-hadoop-1.3.0.M2.jar
to class path
Added resource: /home/hduser/hadoop/

elasticsearch-hadoop-1.3.0.M2.__jar

    hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
    Total MapReduce jobs = 1
    Launching Job 1 out of 1
    Number of reduce tasks is set to 0 since there's no reduce

operator
java.io.FileNotFoundException: File does not exist:
/home/hduser/hadoop/__elasticsearch-hadoop-1.3.0.M2.__jar
at org.apache.hadoop.hdfs.DistributedFileSystem.
getFileStatus(__DistributedFileSystem.java:__558)
at org.apache.hadoop.filecache.DistributedCache.
getFileStatus(__DistributedCache.java:185)
at
org.apache.hadoop.filecache.__TrackerDistributedCacheManager
.getFileStatus(TrackerDistributedCacheManager.java:723)
at
org.apache.hadoop.filecache.TrackerDistributedCacheManager
.determineTimestamps(TrackerDistributedCacheManager.java:778)
at
org.apache.hadoop.filecache.TrackerDistributedCacheManager.

determineTimestampsAndCacheVis__ibilities(

TrackerDistributedCacheManager
.java:755)
at org.apache.hadoop.mapred.JobClient.
copyAndConfigureFiles(__JobClient.java:843)
at org.apache.hadoop.mapred.JobClient.
copyAndConfigureFiles(__JobClient.java:734)
at org.apache.hadoop.mapred.JobClient.access$400(
JobClient.java:179)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.
java:951)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.
java:936)
at java.security.__AccessController.doPrivileged(__Native
Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.

UserGroupInformation.doAs(__UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.

JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapred.

JobClient.submitJob(JobClient.java:910)
at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(
ExecDriver.java:447)
at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(
MapRedTask.java:138)
at org.apache.hadoop.hive.ql.

exec.Task.executeTask(Task.java:144)
at org.apache.hadoop.hive.ql.

exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.

Driver.launchTask(Driver.java:1355)
at org.apache.hadoop.hive.ql.

Driver.execute(Driver.java:__1139)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
at org.apache.hadoop.hive.cli.

CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(
CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(
CliDriver.java:413)
at org.apache.hadoop.hive.cli.

CliDriver.run(CliDriver.java:756)
at org.apache.hadoop.hive.cli.

CliDriver.main(CliDriver.java:__614)
at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native
Method)
at sun.reflect.__NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:__57)
at sun.reflect.__DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.__java:43)
at java.lang.reflect.Method.__invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.__main(RunJar.java:160)
Job Submission failed with exception 'java.io http://java.io.__FileNotFoundException(File
does not exist:
/home/hduser/hadoop/__elasticsearch-hadoop-1.3.0.M2.__jar)'
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.__exec.MapRedTask

    That path it exist :/home/hduser/hadoop/__

elasticsearch-hadoop-1.3.0.M2.__jar

    Any help would be so appreciable
    Thanks

    --
    You received this message because you are subscribed to the

Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>>.

    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/d1613b48-_

_e58d-4077-a579-92b46a97bcff%__40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/d1613b48-
e58d-4077-a579-92b46a97bcff%40googlegroups.com>
<https://groups.google.com/d/__msgid/elasticsearch/d1613b48-
__e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/d1613b48-
e58d-4077-a579-92b46a97bcff%40googlegroups.com?utm_medium=
email&utm_source=footer>>.

    For more options, visit https://groups.google.com/d/__optout <

https://groups.google.com/d/optout>.

--
Costin


--
You received this message because you are subscribed to the Google

Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%
2Bunsubscribe@googlegroups.com>.

To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/537B549F._

_8020501%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/537B549F.
8020501%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <

https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-
xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-
xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/537C738E.9030001%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANXJSR8Gy4qhM8Lzm5BHCr2-xptLKBSdm00nCrz-JtnW59jWcw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Costin Leau) #6

If you plan to use es-hadoop, there's no need to do any JSON conversion (es-hadoop does that automatically) - see the
documentation at [1]
for more information.
If not than I recommend reaching out to the providers of the package for more information on its configuration.

[1] elasticsearch.org/hadoop

On 5/21/14 1:17 PM, hanine haninne wrote:

yes, you are right.Thanks so much

well I tried hive with HDFS and it works very well,by the way I m using hive-0.11.0 , hadoop-1.2.1 and elasticsearch-1.0.1
Here is the script of the tables
CREATE EXTERNAL TABLE tweets (
id BIGINT,
created_at STRING,
source STRING,
favorited BOOLEAN,
retweet_count INT,
retweeted_status STRUCT<
text:STRING,
user:STRUCT<screen_name:STRING,name:STRING>>,
entities STRUCT<
urls:ARRAY<STRUCT<expanded_url:STRING>>,
user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
hashtags:ARRAY<STRUCTtext:STRING>>,
text STRING,
user STRUCT<
screen_name:STRING,
name:STRING,
friends_count:INT,
followers_count:INT,
statuses_count:INT,
verified:BOOLEAN,
utc_offset:INT,
time_zone:STRING>,
in_reply_to_screen_name STRING
)
ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe'
LOCATION '/flume/Tweets_A';

do I have to add "com.cloudera.hive.serde.JSONSerDe'" an other time to put data in elasticsearch ?? if yes,How !!

Thank you so very much

2014-05-21 10:36 GMT+01:00 Costin Leau <costin.leau@gmail.com mailto:costin.leau@gmail.com>:

The error indicates that your script refers to a JSonSerDe class (presumably from Cloudera) which cannot be found in
the classpath:

"> Caused by: java.lang.__ClassNotFoundException: com.cloudera.hive.serde.__JSONSerDe"

In fact if you read through the stacktrace, you'll notice that this comes up while the job configuration
is being read and this class name encountered but the class definition itself cannot be resolved.

I'm not sure what causes this class to be added to the classpath of your script - it could be your script or the
client configuration.
Note the class is not used or added by ES-hadoop itself.

I'm not sure what distro you are using (es-hadoop works across Apache Hadoop, Cloudera, HDP, MapR, Pivotal, etc...)
but you could try running some
basic Hive scripts (without es-hadoop) first to make sure the environment is sane and that you can properly read and
write from HDFS and then add
es-hadoop into the mix.

Hope this helps,


On 5/21/14 12:15 PM, hanine haninne wrote:

    Thank you so much for ur answer
    here is what it gives me now

        INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;

    Total MapReduce jobs = 1
    Launching Job 1 out of 1
    Number of reduce tasks is set to 0 since there's no reduce operator
    Starting Job = job_201405210900_0002, Tracking URL =
    http://master:50030/__jobdetails.jsp?jobid=job___201405210900_0002
    <http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002>
    Kill Command = /home/hduser/hadoop/libexec/..__/bin/hadoop job  -kill job_201405210900_0002
    Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0
    2014-05-21 10:12:05,285 Stage-0 map = 0%,  reduce = 0%
    2014-05-21 10:12:23,408 Stage-0 map = 100%,  reduce = 100%
    Ended Job = job_201405210900_0002 with errors
    Error during job, obtaining debugging information...
    Job Tracking URL: http://master:50030/__jobdetails.jsp?jobid=job___201405210900_0002
    <http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002>
    Examining task ID: task_201405210900_0002_m___000002 (and more) from job job_201405210900_0002

    Task with the most failures(4):
    -----
    Task ID:
        task_201405210900_0002_m___000000

    URL:
    http://master:50030/__taskdetails.jsp?jobid=job___201405210900_0002&tipid=task___201405210900_0002_m_000000
    <http://master:50030/taskdetails.jsp?jobid=job_201405210900_0002&tipid=task_201405210900_0002_m_000000>
    -----
    Diagnostic Messages for this Task:
    java.lang.RuntimeException: Error in configuring object
          at org.apache.hadoop.util.__ReflectionUtils.setJobConf(__ReflectionUtils.java:93)
          at org.apache.hadoop.util.__ReflectionUtils.setConf(__ReflectionUtils.java:64)
          at org.apache.hadoop.util.__ReflectionUtils.newInstance(__ReflectionUtils.java:117)
          at org.apache.hadoop.mapred.__MapTask.runOldMapper(MapTask.__java:426)
          at org.apache.hadoop.mapred.__MapTask.run(MapTask.java:366)
          at org.apache.hadoop.mapred.__Child$4.run(Child.java:255)
          at java.security.__AccessController.doPrivileged(__Native Method)
          at javax.security.auth.Subject.__doAs(Subject.java:415)
          at org.apache.hadoop.security.__UserGroupInformation.doAs(__UserGroupInformation.java:__1190)
          at org.apache.hadoop.mapred.__Child.main(Child.java:249)
    Caused by: java.lang.reflect.__InvocationTargetException
          at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native Method)
          at sun.reflect.__NativeMethodAccessorImpl.__invoke(__NativeMethodAccessorImpl.java:__57)
          at sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__DelegatingMethodAccessorImpl.__java:43)
          at java.lang.reflect.Method.__invoke(Method.java:606)
          at org.apache.hadoop.util.__ReflectionUtils.setJobConf(__ReflectionUtils.java:88)
          ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
          at org.apache.hadoop.util.__ReflectionUtils.setJobConf(__ReflectionUtils.java:93)
          at org.apache.hadoop.util.__ReflectionUtils.setConf(__ReflectionUtils.java:64)
          at org.apache.hadoop.util.__ReflectionUtils.newInstance(__ReflectionUtils.java:117)
          at org.apache.hadoop.mapred.__MapRunner.configure(MapRunner.__java:34)
          ... 14 more
    Caused by: java.lang.reflect.__InvocationTargetException
          at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native Method)
          at sun.reflect.__NativeMethodAccessorImpl.__invoke(__NativeMethodAccessorImpl.java:__57)
          at sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__DelegatingMethodAccessorImpl.__java:43)
          at java.lang.reflect.Method.__invoke(Method.java:606)
          at org.apache.hadoop.util.__ReflectionUtils.setJobConf(__ReflectionUtils.java:88)
          ... 17 more
    Caused by: java.lang.RuntimeException: Map operator initialization failed
          at org.apache.hadoop.hive.ql.__exec.ExecMapper.configure(__ExecMapper.java:121)
          ... 22 more
    Caused by: org.apache.hadoop.hive.ql.__metadata.HiveException: java.lang.__ClassNotFoundException:
    com.cloudera.hive.serde.__JSONSerDe
          at org.apache.hadoop.hive.ql.__exec.MapOperator.__getConvertedOI(MapOperator.__java:463)
          at org.apache.hadoop.hive.ql.__exec.MapOperator.setChildren(__MapOperator.java:479)
          at org.apache.hadoop.hive.ql.__exec.ExecMapper.configure(__ExecMapper.java:90)
          ... 22 more
    Caused by: java.lang.__ClassNotFoundException: com.cloudera.hive.serde.__JSONSerDe
          at java.net.URLClassLoader$1.run(__URLClassLoader.java:366)
          at java.net.URLClassLoader$1.run(__URLClassLoader.java:355)
          at java.security.__AccessController.doPrivileged(__Native Method)
          at java.net.URLClassLoader.__findClass(URLClassLoader.java:__354)
          at java.lang.ClassLoader.__loadClass(ClassLoader.java:__424)
          at sun.misc.Launcher$__AppClassLoader.loadClass(__Launcher.java:308)
          at java.lang.ClassLoader.__loadClass(ClassLoader.java:__357)
          at java.lang.Class.forName0(__Native Method)
          at java.lang.Class.forName(Class.__java:270)
          at org.apache.hadoop.conf.__Configuration.getClassByName(__Configuration.java:810)
          at org.apache.hadoop.hive.ql.__exec.MapOperator.__getConvertedOI(MapOperator.__java:422)
          ... 24 more


    FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.__exec.MapRedTask
    MapReduce Jobs Launched:
    Job 0: Map: 1   HDFS Read: 0 HDFS Write: 0 FAIL
    Total MapReduce CPU Time Spent: 0 msec



    2014-05-20 14:11 GMT+01:00 Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__>:


         1. You don't need to add the hive jars (hive-serdes) to your script - these are already part of the runtime
         2. use es-hadoop-2.0.RC1, 1.3 M2 is fairly old
         3. Make sure that the jar is actually available and that is potentially accessible by Hive. Also do note that
         when running Hive against a cluster, it's best to have the jars available on all nodes hence why most folks
    copy
         them to HDFS and refer to that location.
         While /home/hduser might be locally available, it might not exist on the Hive server - as indicated by the
    error.

         Hope this helps,


         On 5/20/14 3:58 PM, hanine haninne wrote:


             Hello everybody

             I m trying to take data from Hive and put it in elasticsearch
             here is the script and the error

             hive> ADD JAR /home/hduser/hadoop/hive-0.11.____0/lib/hive-serdes-1.0-____SNAPSHOT.jar;
             Added /home/hduser/hadoop/hive-0.11.____0/lib/hive-serdes-1.0-____SNAPSHOT.jar to class path
             Added resource: /home/hduser/hadoop/hive-0.11.____0/lib/hive-serdes-1.0-____SNAPSHOT.jar

             hive> CREATE EXTERNAL TABLE tweetsES (
                   >    id BIGINT,
                   >    created_at STRING,
                   >    source STRING,
                   >    favorited BOOLEAN,
                   >    retweet_count INT,
                   >    retweeted_status STRUCT<
                   >       text:STRING,
                   >       user:STRUCT<screen_name:____STRING,name:STRING>>,
                   >    entities STRUCT<
                   >       urls:ARRAY<STRUCT<expanded_____url:STRING>>,
                   >       user_mentions:ARRAY<STRUCT<____screen_name:STRING,name:____STRING>>,
                   >       hashtags:ARRAY<STRUCT<text:____STRING>>>,

                   >    text STRING,
                   >    user STRUCT<
                   >       screen_name:STRING,
                   >       name:STRING,
                   >       friends_count:INT,
                   >       followers_count:INT,
                   >       statuses_count:INT,
                   >       verified:BOOLEAN,
                   >       utc_offset:INT,
                   >       time_zone:STRING>,
                   >    in_reply_to_screen_name STRING
                   > )
                   > STORED BY 'org.elasticsearch.hadoop.____hive.EsStorageHandler'
                   > TBLPROPERTIES('es.resource' = 'reseaux_sociaux/tweets','es.____input.json' = 'yes');

             OK
             Time taken: 4.292 seconds
             hive> ADD JAR /home/hduser/hadoop/____elasticsearch-hadoop-1.3.0.M2.____jar;
             Added /home/hduser/hadoop/____elasticsearch-hadoop-1.3.0.M2.____jar to class path
             Added resource: /home/hduser/hadoop/____elasticsearch-hadoop-1.3.0.M2.____jar

             hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
             Total MapReduce jobs = 1
             Launching Job 1 out of 1
             Number of reduce tasks is set to 0 since there's no reduce operator
             java.io.FileNotFoundException: File does not exist:
    /home/hduser/hadoop/____elasticsearch-hadoop-1.3.0.M2.____jar
                   at
    org.apache.hadoop.hdfs.____DistributedFileSystem.____getFileStatus(____DistributedFileSystem.java:____558)
                   at org.apache.hadoop.filecache.____DistributedCache.____getFileStatus(____DistributedCache.java:185)
                   at

    org.apache.hadoop.filecache.____TrackerDistributedCacheManager____.getFileStatus(____TrackerDistributedCacheManager____.java:723)
                   at

    org.apache.hadoop.filecache.____TrackerDistributedCacheManager____.determineTimestamps(____TrackerDistributedCacheManager____.java:778)
                   at

    org.apache.hadoop.filecache.____TrackerDistributedCacheManager____.____determineTimestampsAndCacheVis____ibilities(____TrackerDistributedCacheManager____.java:755)
                   at org.apache.hadoop.mapred.____JobClient.____copyAndConfigureFiles(____JobClient.java:843)
                   at org.apache.hadoop.mapred.____JobClient.____copyAndConfigureFiles(____JobClient.java:734)
                   at org.apache.hadoop.mapred.____JobClient.access$400(____JobClient.java:179)
                   at org.apache.hadoop.mapred.____JobClient$2.run(JobClient.____java:951)
                   at org.apache.hadoop.mapred.____JobClient$2.run(JobClient.____java:936)
                   at java.security.____AccessController.doPrivileged(____Native Method)
                   at javax.security.auth.Subject.____doAs(Subject.java:415)
                   at org.apache.hadoop.security.____UserGroupInformation.doAs(____UserGroupInformation.java:____1190)
                   at org.apache.hadoop.mapred.____JobClient.submitJobInternal(____JobClient.java:936)
                   at org.apache.hadoop.mapred.____JobClient.submitJob(JobClient.____java:910)
                   at org.apache.hadoop.hive.ql.____exec.ExecDriver.execute(____ExecDriver.java:447)
                   at org.apache.hadoop.hive.ql.____exec.MapRedTask.execute(____MapRedTask.java:138)
                   at org.apache.hadoop.hive.ql.____exec.Task.executeTask(Task.____java:144)
                   at org.apache.hadoop.hive.ql.____exec.TaskRunner.runSequential(____TaskRunner.java:57)
                   at org.apache.hadoop.hive.ql.____Driver.launchTask(Driver.java:____1355)
                   at org.apache.hadoop.hive.ql.____Driver.execute(Driver.java:____1139)
                   at org.apache.hadoop.hive.ql.____Driver.run(Driver.java:945)
                   at org.apache.hadoop.hive.cli.____CliDriver.processLocalCmd(____CliDriver.java:259)
                   at org.apache.hadoop.hive.cli.____CliDriver.processCmd(____CliDriver.java:216)
                   at org.apache.hadoop.hive.cli.____CliDriver.processLine(____CliDriver.java:413)
                   at org.apache.hadoop.hive.cli.____CliDriver.run(CliDriver.java:____756)
                   at org.apache.hadoop.hive.cli.____CliDriver.main(CliDriver.java:____614)
                   at sun.reflect.____NativeMethodAccessorImpl.____invoke0(Native Method)
                   at sun.reflect.____NativeMethodAccessorImpl.____invoke(____NativeMethodAccessorImpl.java:____57)
                   at
    sun.reflect.____DelegatingMethodAccessorImpl.____invoke(____DelegatingMethodAccessorImpl.____java:43)
                   at java.lang.reflect.Method.____invoke(Method.java:606)
                   at org.apache.hadoop.util.RunJar.____main(RunJar.java:160)
             Job Submission failed with exception 'java.io <http://java.io>
    <http://java.io>.____FileNotFoundException(File does not exist:
             /home/hduser/hadoop/____elasticsearch-hadoop-1.3.0.M2.____jar)'
             FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.____exec.MapRedTask


             That path it exist :/home/hduser/hadoop/____elasticsearch-hadoop-1.3.0.M2.____jar

             Any help would be so appreciable
             Thanks

             --
             You received this message because you are subscribed to the Google Groups "elasticsearch" group.
             To unsubscribe from this group and stop receiving emails from it, send an email to
             elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
    <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>
             <mailto:elasticsearch+____unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B__unsubscribe@googlegroups.com> <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>>.


             To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-__e58d-4077-a579-92b46a97bcff%__40googlegroups.com>

    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-__e58d-4077-a579-92b46a97bcff%__40googlegroups.com
    <https://groups.google.com/d/msgid/elasticsearch/d1613b48-e58d-4077-a579-92b46a97bcff%40googlegroups.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-__e58d-4077-a579-92b46a97bcff%__40googlegroups.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-__e58d-4077-a579-92b46a97bcff%__40googlegroups.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/d1613b48-e58d-4077-a579-92b46a97bcff%40googlegroups.com?utm_medium=email&utm_source=footer>>>.

             For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.



         --
         Costin


         --
         You received this message because you are subscribed to the Google Groups "elasticsearch" group.
         To unsubscribe from this group and stop receiving emails from it, send an email to
         elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
    <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>.

         To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/537B549F.____8020501%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/537B549F.__8020501%40gmail.com>
         <https://groups.google.com/d/__msgid/elasticsearch/537B549F.__8020501%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/537B549F.8020501%40gmail.com>>.

         For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.



    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>>.
    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-__xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%__40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%40mail.gmail.com>
    <https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-__xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%__40mail.gmail.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%40mail.gmail.com?utm_medium=email&utm_source=footer>>.

    For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.


--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/537C738E.__9030001%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/537C738E.9030001%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANXJSR8Gy4qhM8Lzm5BHCr2-xptLKBSdm00nCrz-JtnW59jWcw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANXJSR8Gy4qhM8Lzm5BHCr2-xptLKBSdm00nCrz-JtnW59jWcw%40mail.gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/537C8751.1030908%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


(hanine) #7

Hello again,
I m trying to get some tweets from the HDFS and put it in
elasticsearch(head) in order to make search on it, I did what u said and I
m so thankful, But it start reading and does not continue. I don't know why
?!!
here is what I did :

hive -hiveconf
hive.aux.jars.path=/ES_Hadoop/dist/elasticsearch-hadoop-2.0.0.RC1.jar

Logging initialized using configuration in
jar:file:/home/hduser/hadoop/hive-0.11.0/lib/hive-common-0.11.0.jar!/hive-log4j.properties
Hive history
file=/tmp/hduser/hive_job_log_hduser_8166@mouna-pc_201405220958_2036150447.txt

hive> ADD JAR
/home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar;
Added /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar to
class path
Added resource:
/home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar
hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM
tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201405220937_0005, Tracking URL =
http://master:50030/jobdetails.jsp?jobid=job_201405220937_0005
Kill Command = /home/hduser/hadoop/libexec/../bin/hadoop job -kill
job_201405220937_0005
Hadoop job information for Stage-0: number of mappers: 1; number of
reducers: 0
2014-05-22 10:01:39,184 Stage-0 map = 0%, reduce = 0%
2014-05-22 10:01:59,320 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201405220937_0005 with errors
Error during job, obtaining debugging information...
Job Tracking URL:
http://master:50030/jobdetails.jsp?jobid=job_201405220937_0005
Examining task ID: task_201405220937_0005_m_000002 (and more) from job
job_201405220937_0005

Task with the most failures(4):

Task ID:
task_201405220937_0005_m_000000

URL:

http://master:50030/taskdetails.jsp?jobid=job_201405220937_0005&tipid=task_201405220937_0005_m_000000

Diagnostic Messages for this Task:
java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {"id":467649114269028352,"created_at":"Sat May 17 12:53:32
+0000 2014","source":"<a href="http://www.apple.com"
rel="nofollow">iOS","favorited":false,"retweet_count":0,"retweeted_status":null,"entities":{"urls":[{"expanded_url":"
https://www.google.co.uk/search?q=viera+lifting+fa+cup+for+arsenal&client=safari&hl=en-gb&source=lnms&tbm=isch&sa=X&ei=alp3U4LrK82UOufRgaAK&ved=0CAYQ_AUoAQ&biw=320&bih=460#facrc=&imgrc=-QK5IhwQT8AS8M%253A%3B2yfUWG7-cngS-M%3Bhttp%253A%252F%252Ffarm4.static.flickr.com%252F3777%252F10838930556_618b87d7cc_m.jpg%3Bhttp%253A%252F%252Fwhotalking.com%252Fflickr%252F2-0%252BChelsea%3B1024%3B669"}],"user_mentions":[{"screen_name":"jameswilliams46","name":"James
Williams"}],"hashtags":[]},"text":"@jameswilliams46
https://t.co/CAmxdagAym","user":{"screen_name":"SSimpson7379","name":"Stephen
Simpson","friends_count":258,"followers_count":266,"statuses_count":2609,"verified":false,"utc_offset":3600,"time_zone":"London"},"in_reply_to_screen_name":"jameswilliams46"}
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:162)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row {"id":467649114269028352,"created_at":"Sat May
17 12:53:32 +0000 2014","source":"<a href="http://www.apple.com"
rel="nofollow">iOS","favorited":false,"retweet_count":0,"retweeted_status":null,"entities":{"urls":[{"expanded_url":"
https://www.google.co.uk/search?q=viera+lifting+fa+cup+for+arsenal&client=safari&hl=en-gb&source=lnms&tbm=isch&sa=X&ei=alp3U4LrK82UOufRgaAK&ved=0CAYQ_AUoAQ&biw=320&bih=460#facrc=
&imgrc=-QK5IhwQT8AS8M%253A%3B2yfUWG7-cngS-M%3Bhttp%253A%252F%252Ffarm4.static.flickr.com%252F3777%252F10838930556_618b87d7cc_m.jpg%3Bhttp%253A%252F%252Fwhotalking.com%252Fflickr%252F2-0%252BChelsea%3B1024%3B669"}],"user_mentions":[{"screen_name":"jameswilliams46","name":"James
Williams"}],"hashtags":[]},"text":"@jameswilliams46
https://t.co/CAmxdagAym","user":{"screen_name":"SSimpson7379","name":"Stephen
Simpson","friends_count":258,"followers_count":266,"statuses_count":2609,"verified":false,"utc_offset":3600,"time_zone":"London"},"in_reply_to_screen_name":"jameswilliams46"}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:671)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144)
... 8 more
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: When
using JSON input, only one field is expected
at org.elasticsearch.hadoop.util.Assert.isTrue(Assert.java:50)
at
org.elasticsearch.hadoop.hive.HiveBytesConverter.convert(HiveBytesConverter.java:48)
at
org.elasticsearch.hadoop.serialization.bulk.JsonTemplatedBulk.preProcess(JsonTemplatedBulk.java:58)
at
org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.write(TemplatedBulk.java:75)
at org.elasticsearch.hadoop.hive.EsSerDe.serialize(EsSerDe.java:128)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:617)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:652)
... 9 more

FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

when I did not add that jar "ADD JAR
/home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar;" it gave
me that error "Caused by: java.lang.ClassNotFoundException:
com.cloudera.hive.serde.JSONSerDe
"

Thank you so very much for your helping

2014-05-21 12:00 GMT+01:00 Costin Leau costin.leau@gmail.com:

If you plan to use es-hadoop, there's no need to do any JSON conversion
(es-hadoop does that automatically) - see the documentation at [1]
for more information.
If not than I recommend reaching out to the providers of the package for
more information on its configuration.

[1] elasticsearch.org/hadoop

On 5/21/14 1:17 PM, hanine haninne wrote:

yes, you are right.Thanks so much

well I tried hive with HDFS and it works very well,by the way I m using
hive-0.11.0 , hadoop-1.2.1 and elasticsearch-1.0.1
Here is the script of the tables
CREATE EXTERNAL TABLE tweets (
id BIGINT,
created_at STRING,
source STRING,
favorited BOOLEAN,
retweet_count INT,
retweeted_status STRUCT<
text:STRING,
user:STRUCT<screen_name:STRING,name:STRING>>,
entities STRUCT<
urls:ARRAY<STRUCT<expanded_url:STRING>>,
user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
hashtags:ARRAY<STRUCTtext:STRING>>,
text STRING,
user STRUCT<
screen_name:STRING,
name:STRING,
friends_count:INT,
followers_count:INT,
statuses_count:INT,
verified:BOOLEAN,
utc_offset:INT,
time_zone:STRING>,
in_reply_to_screen_name STRING
)
ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe'
LOCATION '/flume/Tweets_A';

do I have to add "com.cloudera.hive.serde.JSONSerDe'" an other time to
put data in elasticsearch ?? if yes,How !!

Thank you so very much

2014-05-21 10:36 GMT+01:00 Costin Leau <costin.leau@gmail.com <mailto:
costin.leau@gmail.com>>:

The error indicates that your script refers to a JSonSerDe class

(presumably from Cloudera) which cannot be found in
the classpath:

"> Caused by: java.lang.__ClassNotFoundException:

com.cloudera.hive.serde.__JSONSerDe"

In fact if you read through the stacktrace, you'll notice that this

comes up while the job configuration
is being read and this class name encountered but the class
definition itself cannot be resolved.

I'm not sure what causes this class to be added to the classpath of

your script - it could be your script or the
client configuration.
Note the class is not used or added by ES-hadoop itself.

I'm not sure what distro you are using (es-hadoop works across Apache

Hadoop, Cloudera, HDP, MapR, Pivotal, etc...)
but you could try running some
basic Hive scripts (without es-hadoop) first to make sure the
environment is sane and that you can properly read and
write from HDFS and then add
es-hadoop into the mix.

Hope this helps,


On 5/21/14 12:15 PM, hanine haninne wrote:

    Thank you so much for ur answer
    here is what it gives me now

        INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;

    Total MapReduce jobs = 1
    Launching Job 1 out of 1
    Number of reduce tasks is set to 0 since there's no reduce

operator
Starting Job = job_201405210900_0002, Tracking URL =
http://master:50030/jobdetails.jsp?jobid=job_
201405210900_0002
http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002
Kill Command = /home/hduser/hadoop/libexec/..__/bin/hadoop job
-kill job_201405210900_0002

    Hadoop job information for Stage-0: number of mappers: 1; number

of reducers: 0
2014-05-21 10:12:05,285 Stage-0 map = 0%, reduce = 0%
2014-05-21 10:12:23,408 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201405210900_0002 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://master:50030/__
jobdetails.jsp?jobid=job___201405210900_0002
http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002
Examining task ID: task_201405210900_0002_m___000002 (and more)
from job job_201405210900_0002

    Task with the most failures(4):
    -----
    Task ID:
        task_201405210900_0002_m___000000

    URL:
    http://master:50030/__taskdetails.jsp?jobid=job___

201405210900_0002&tipid=task___201405210900_0002_m_000000

    <http://master:50030/taskdetails.jsp?jobid=job_

201405210900_0002&tipid=task_201405210900_0002_m_000000>
-----
Diagnostic Messages for this Task:
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(
ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(
ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(
ReflectionUtils.java:117)
at org.apache.hadoop.mapred.__
MapTask.runOldMapper(MapTask.__java:426)
at org.apache.hadoop.mapred.__MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.__Child$4.run(Child.java:255)

          at java.security.__AccessController.doPrivileged(__Native

Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.

UserGroupInformation.doAs(__UserGroupInformation.java:__1190)
at org.apache.hadoop.mapred.__Child.main(Child.java:249)
Caused by: java.lang.reflect.__InvocationTargetException

          at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native

Method)
at sun.reflect.__NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:__57)
at sun.reflect.__DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.__java:43)
at java.lang.reflect.Method.__invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(
ReflectionUtils.java:88)

          ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
          at org.apache.hadoop.util.__ReflectionUtils.setJobConf(__

ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(
ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(
ReflectionUtils.java:117)
at org.apache.hadoop.mapred.__
MapRunner.configure(MapRunner.__java:34)
... 14 more
Caused by: java.lang.reflect.__InvocationTargetException

          at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native

Method)
at sun.reflect.__NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:__57)
at sun.reflect.__DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.__java:43)
at java.lang.reflect.Method.__invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(
ReflectionUtils.java:88)

          ... 17 more
    Caused by: java.lang.RuntimeException: Map operator

initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(
ExecMapper.java:121)
... 22 more
Caused by: org.apache.hadoop.hive.ql.__metadata.HiveException:
java.lang.__ClassNotFoundException:
com.cloudera.hive.serde.__JSONSerDe
at org.apache.hadoop.hive.ql.exec.MapOperator.
getConvertedOI(MapOperator.java:463)
at org.apache.hadoop.hive.ql.

exec.MapOperator.setChildren(__MapOperator.java:479)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(
ExecMapper.java:90)
... 22 more
Caused by: java.lang.__ClassNotFoundException:
com.cloudera.hive.serde.__JSONSerDe
at java.net.URLClassLoader$1.run(__URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(__URLClassLoader.java:355)
at java.security.__AccessController.doPrivileged(__Native
Method)
at java.net.URLClassLoader.__findClass(URLClassLoader.java:
__354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:
424)
at sun.misc.Launcher$AppClassLoader.loadClass(
Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:
357)
at java.lang.Class.forName0(__Native Method)
at java.lang.Class.forName(Class.__java:270)
at org.apache.hadoop.conf._Configuration.getClassByName(
_Configuration.java:810)
at org.apache.hadoop.hive.ql.exec.MapOperator.
getConvertedOI(MapOperator.__java:422)
... 24 more

    FAILED: Execution Error, return code 2 from

org.apache.hadoop.hive.ql.__exec.MapRedTask

    MapReduce Jobs Launched:
    Job 0: Map: 1   HDFS Read: 0 HDFS Write: 0 FAIL
    Total MapReduce CPU Time Spent: 0 msec



    2014-05-20 14:11 GMT+01:00 Costin Leau <costin.leau@gmail.com<mailto:

costin.leau@gmail.com>
<mailto:costin.leau@gmail.com mailto:costin.leau@gmail.com>__>:

         1. You don't need to add the hive jars (hive-serdes) to your

script - these are already part of the runtime
2. use es-hadoop-2.0.RC1, 1.3 M2 is fairly old
3. Make sure that the jar is actually available and that is
potentially accessible by Hive. Also do note that
when running Hive against a cluster, it's best to have the
jars available on all nodes hence why most folks
copy
them to HDFS and refer to that location.
While /home/hduser might be locally available, it might not
exist on the Hive server - as indicated by the
error.

         Hope this helps,


         On 5/20/14 3:58 PM, hanine haninne wrote:


             Hello everybody

             I m trying to take data from Hive and put it in

elasticsearch
here is the script and the error

             hive> ADD JAR /home/hduser/hadoop/hive-0.11.

____0/lib/hive-serdes-1.0-____SNAPSHOT.jar;
Added /home/hduser/hadoop/hive-0.11.
____0/lib/hive-serdes-1.0-____SNAPSHOT.jar to class path
Added resource: /home/hduser/hadoop/hive-0.11.
____0/lib/hive-serdes-1.0-____SNAPSHOT.jar

             hive> CREATE EXTERNAL TABLE tweetsES (
                   >    id BIGINT,
                   >    created_at STRING,
                   >    source STRING,
                   >    favorited BOOLEAN,
                   >    retweet_count INT,
                   >    retweeted_status STRUCT<
                   >       text:STRING,
                   >       user:STRUCT<screen_name:____

STRING,name:STRING>>,
> entities STRUCT<
> urls:ARRAY<STRUCT<expanded____
url:STRING>>,
> user_mentions:ARRAY<STRUCT<
__
_screen_name:STRING,name:____STRING>>,
> hashtags:ARRAY<STRUCTtext:____STRING>>,

                   >    text STRING,
                   >    user STRUCT<
                   >       screen_name:STRING,
                   >       name:STRING,
                   >       friends_count:INT,
                   >       followers_count:INT,
                   >       statuses_count:INT,
                   >       verified:BOOLEAN,
                   >       utc_offset:INT,
                   >       time_zone:STRING>,
                   >    in_reply_to_screen_name STRING
                   > )
                   > STORED BY 'org.elasticsearch.hadoop.____

hive.EsStorageHandler'
> TBLPROPERTIES('es.resource' =
'reseaux_sociaux/tweets','es.____input.json' = 'yes');

             OK
             Time taken: 4.292 seconds
             hive> ADD JAR /home/hduser/hadoop/____

elasticsearch-hadoop-1.3.0.M2.jar;
Added /home/hduser/hadoop/

elasticsearch-hadoop-1.3.0.M2.jar to class path
Added resource: /home/hduser/hadoop/

elasticsearch-hadoop-1.3.0.M2.____jar

             hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM

tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no
reduce operator
java.io.FileNotFoundException: File does not exist:
/home/hduser/hadoop/____elasticsearch-hadoop-1.3.0.M2.____jar
at
org.apache.hadoop.hdfs.DistributedFileSystem.
getFileStatus(____DistributedFileSystem.java:__558)
at org.apache.hadoop.filecache.

__DistributedCache.____getFileStatus(____DistributedCache.java:185)
at

    org.apache.hadoop.filecache.____TrackerDistributedCacheManager

____.getFileStatus(TrackerDistributedCacheManager.java:723)
at

    org.apache.hadoop.filecache.____TrackerDistributedCacheManager

____.determineTimestamps(TrackerDistributedCacheManager.java:778)
at

    org.apache.hadoop.filecache.____TrackerDistributedCacheManager

.determineTimestampsAndCacheVis____ibilities(
TrackerDistributedCacheManager
.java:755)
at org.apache.hadoop.mapred.JobClient.
copyAndConfigureFiles(____JobClient.java:843)
at org.apache.hadoop.mapred.JobClient.
copyAndConfigureFiles(JobClient.java:734)
at org.apache.hadoop.mapred.

JobClient.access$400(JobClient.java:179)
at org.apache.hadoop.mapred.

JobClient$2.run(JobClient.java:951)
at org.apache.hadoop.mapred.

JobClient$2.run(JobClient.java:936)
at java.security.

AccessController.doPrivileged(_Native Method)
at javax.security.auth.Subject.

doAs(Subject.java:415)
at org.apache.hadoop.security.

_UserGroupInformation.doAs(_UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.

JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapred.

JobClient.submitJob(JobClient.java:910)
at org.apache.hadoop.hive.ql.

exec.ExecDriver.execute(ExecDriver.java:447)
at org.apache.hadoop.hive.ql.

exec.MapRedTask.execute(MapRedTask.java:138)
at org.apache.hadoop.hive.ql.

exec.Task.executeTask(Task.java:144)
at org.apache.hadoop.hive.ql.

exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.

Driver.launchTask(Driver.java:1355)
at org.apache.hadoop.hive.ql.

Driver.execute(Driver.java:1139)
at org.apache.hadoop.hive.ql.

Driver.run(Driver.java:945)
at org.apache.hadoop.hive.cli.

_CliDriver.processLocalCmd(_CliDriver.java:259)
at org.apache.hadoop.hive.cli.

_CliDriver.processCmd(_CliDriver.java:216)
at org.apache.hadoop.hive.cli.

_CliDriver.processLine(_CliDriver.java:413)
at org.apache.hadoop.hive.cli.

_CliDriver.run(CliDriver.java:_756)
at org.apache.hadoop.hive.cli.

_CliDriver.main(CliDriver.java:____614)
at sun.reflect.____NativeMethodAccessorImpl.____invoke0(Native
Method)
at sun.reflect.NativeMethodAccessorImpl.
invoke(____NativeMethodAccessorImpl.java:____57)
at
sun.reflect.____DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.

invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.
____main(RunJar.java:160)

             Job Submission failed with exception 'java.io <

http://java.io>
http://java.io.____FileNotFoundException(File does not exist:
/home/hduser/hadoop/____elasticsearch-hadoop-1.3.0.M2.
____jar)'
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.____exec.MapRedTask

             That path it exist :/home/hduser/hadoop/____

elasticsearch-hadoop-1.3.0.M2.____jar

             Any help would be so appreciable
             Thanks

             --
             You received this message because you are subscribed to

the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%252Bunsubscribe@googlegroups.com>
>
<mailto:elasticsearch+____unsubscribe@googlegroups.com
mailto:elasticsearch%2B__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com
>>.

             To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/

d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/d1613b48-
__e58d-4077-a579-92b46a97bcff%__40googlegroups.com>

    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-

__e58d-4077-a579-92b46a97bcff%__40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/d1613b48-
e58d-4077-a579-92b46a97bcff%40googlegroups.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/

d1613b48-____e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/d1613b48-
__e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-

__e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/d1613b48-
e58d-4077-a579-92b46a97bcff%40googlegroups.com?utm_medium=
email&utm_source=footer>>>.

             For more options, visit https://groups.google.com/d/__

__optout https://groups.google.com/d/__optout

    <https://groups.google.com/d/__optout <

https://groups.google.com/d/optout>>.

         --
         Costin


         --
         You received this message because you are subscribed to the

Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%252Bunsubscribe@googlegroups.com>
>.

         To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/

537B549F.____8020501%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/537B549F.
__8020501%40gmail.com>

         <https://groups.google.com/d/__msgid/elasticsearch/537B549F.

__8020501%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/537B549F.
8020501%40gmail.com>>.

         For more options, visit https://groups.google.com/d/__

__optout https://groups.google.com/d/__optout

    <https://groups.google.com/d/__optout <

https://groups.google.com/d/optout>>.

    --
    You received this message because you are subscribed to the

Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/_msgid/elasticsearch/CANXJSR9-
_xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%__40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-
xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%40mail.gmail.com>
<https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-
xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com?utm_medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-
xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com?utm_medium=email&utm_source=footer>>.

    For more options, visit https://groups.google.com/d/__optout <

https://groups.google.com/d/optout>.

--
Costin

--
You received this message because you are subscribed to the Google

Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%
2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/_msgid/elasticsearch/537C738E.
_9030001%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/537C738E.
9030001%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <

https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANXJSR8Gy4qhM8Lzm5BHCr2-
xptLKBSdm00nCrz-JtnW59jWcw%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/
CANXJSR8Gy4qhM8Lzm5BHCr2-xptLKBSdm00nCrz-JtnW59jWcw%
40mail.gmail.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/537C8751.1030908%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANXJSR_pj6i55jwA3wYiGgh1d_q86DpcNRKLZzhk-V97vWFLaQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Costin Leau) #8

Hi,

It looks like you have two tables - one that uses the JSONSerDe from cloudera and another one using es-hadoop.
You configured your es-hadoop table to consider the input as json however it does not receive the proper format (as the
exception indicates).
See this [1] section of the documentation for more information.

Let me restate again that you do not the data to Elasticsearch and es-hadoop to be in JSON format - unless it actually
is in JSON
(as in JSON files are stored in HDFS) simply send whatever data you have in Hive to es-hadoop and it will do the JSON
translation automatically
and typically much more efficient.
Again the docs [2] discuss this in detail. Frankly, I'm not sure why you want to use the JSONSerDe.

To conclude: if the data is in raw format in HDFS, just feed to es-hadoop with the input configured as JSON, if it's in
a different format,
load it in Hive as you would typically do and then insert it into the es-hadoop table.

Hope this helps,

[1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html#writing-json-hive
[2] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html#hive

On 5/22/14 12:13 PM, hanine haninne wrote:

Hello again,
I m trying to get some tweets from the HDFS and put it in elasticsearch(head) in order to make search on it, I did what
u said and I m so thankful, But it start reading and does not continue. I don't know why ?!!
here is what I did :

hive -hiveconf hive.aux.jars.path=/ES_Hadoop/dist/elasticsearch-hadoop-2.0.0.RC1.jar

Logging initialized using configuration in
jar:file:/home/hduser/hadoop/hive-0.11.0/lib/hive-common-0.11.0.jar!/hive-log4j.properties
Hive history file=/tmp/hduser/hive_job_log_hduser_8166@mouna-pc_201405220958_2036150447.txt

hive> ADD JAR /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar;
Added /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar to class path
Added resource: /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar
hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201405220937_0005, Tracking URL = http://master:50030/jobdetails.jsp?jobid=job_201405220937_0005
Kill Command = /home/hduser/hadoop/libexec/../bin/hadoop job -kill job_201405220937_0005
Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0
2014-05-22 10:01:39,184 Stage-0 map = 0%, reduce = 0%
2014-05-22 10:01:59,320 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201405220937_0005 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://master:50030/jobdetails.jsp?jobid=job_201405220937_0005
Examining task ID: task_201405220937_0005_m_000002 (and more) from job job_201405220937_0005

Task with the most failures(4):

Task ID:
task_201405220937_0005_m_000000

URL:
http://master:50030/taskdetails.jsp?jobid=job_201405220937_0005&tipid=task_201405220937_0005_m_000000

Diagnostic Messages for this Task:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
{"id":467649114269028352,"created_at":"Sat May 17 12:53:32 +0000 2014","source":"<a href="http://www.apple.com"
rel="nofollow">iOS","favorited":false,"retweet_count":0,"retweeted_status":null,"entities":{"urls":[{"expanded_url":"https://www.google.co.uk/search?q=viera+lifting+fa+cup+for+arsenal&client=safari&hl=en-gb&source=lnms&tbm=isch&sa=X&ei=alp3U4LrK82UOufRgaAK&ved=0CAYQ_AUoAQ&biw=320&bih=460#facrc=&imgrc=-QK5IhwQT8AS8M%253A%3B2yfUWG7-cngS-M%3Bhttp%253A%252F%252Ffarm4.static.flickr.com%252F3777%252F10838930556_618b87d7cc_m.jpg%3Bhttp%253A%252F%252Fwhotalking.com%252Fflickr%252F2-0%252BChelsea%3B1024%3B669"}],"user_mentions":[{"screen_name":"jameswilliams46","name":"James
Williams"}],"hashtags":[]},"text":"@jameswilliams46
https://t.co/CAmxdagAym","user":{"screen_name":"SSimpson7379","name":"Stephen
Simpson","friends_count":258,"followers_count":266,"statuses_count":2609,"verified":false,"utc_offset":3600,"time_zone":"London"},"in_reply_to_screen_name":"jameswilliams46"}
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:162)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
{"id":467649114269028352,"created_at":"Sat May 17 12:53:32 +0000 2014","source":"<a href="http://www.apple.com"
rel="nofollow">iOS","favorited":false,"retweet_count":0,"retweeted_status":null,"entities":{"urls":[{"expanded_url":"https://www.google.co.uk/search?q=viera+lifting+fa+cup+for+arsenal&client=safari&hl=en-gb&source=lnms&tbm=isch&sa=X&ei=alp3U4LrK82UOufRgaAK&ved=0CAYQ_AUoAQ&biw=320&bih=460#facrc=
&imgrc=-QK5IhwQT8AS8M%253A%3B2yfUWG7-cngS-M%3Bhttp%253A%252F%252Ffarm4.static.flickr.com%252F3777%252F10838930556_618b87d7cc_m.jpg%3Bhttp%253A%252F%252Fwhotalking.com%252Fflickr%252F2-0%252BChelsea%3B1024%3B669"}],"user_mentions":[{"screen_name":"jameswilliams46","name":"James
Williams"}],"hashtags":[]},"text":"@jameswilliams46
https://t.co/CAmxdagAym","user":{"screen_name":"SSimpson7379","name":"Stephen
Simpson","friends_count":258,"followers_count":266,"statuses_count":2609,"verified":false,"utc_offset":3600,"time_zone":"London"},"in_reply_to_screen_name":"jameswilliams46"}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:671)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144)
... 8 more
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: When using JSON input, only one field is expected
at org.elasticsearch.hadoop.util.Assert.isTrue(Assert.java:50)
at org.elasticsearch.hadoop.hive.HiveBytesConverter.convert(HiveBytesConverter.java:48)
at org.elasticsearch.hadoop.serialization.bulk.JsonTemplatedBulk.preProcess(JsonTemplatedBulk.java:58)
at org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.write(TemplatedBulk.java:75)
at org.elasticsearch.hadoop.hive.EsSerDe.serialize(EsSerDe.java:128)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:617)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:652)
... 9 more

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

when I did not add that jar "ADD JAR /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar;" it gave me that
error "Caused by: java.lang.ClassNotFoundException: com.cloudera.hive.serde.JSONSerDe
"

Thank you so very much for your helping

2014-05-21 12:00 GMT+01:00 Costin Leau <costin.leau@gmail.com mailto:costin.leau@gmail.com>:

If you plan to use es-hadoop, there's no need to do any JSON conversion (es-hadoop does that automatically) - see
the documentation at [1]
for more information.
If not than I recommend reaching out to the providers of the package for more information on its configuration.

[1] elasticsearch.org/hadoop <http://elasticsearch.org/hadoop>


On 5/21/14 1:17 PM, hanine haninne wrote:

    yes, you are right.Thanks so much

    well I tried hive with HDFS and it works very well,by the way I m using hive-0.11.0 , hadoop-1.2.1 and
    elasticsearch-1.0.1
    Here is the script of the tables
    CREATE EXTERNAL TABLE tweets (
         id BIGINT,
         created_at STRING,
         source STRING,
         favorited BOOLEAN,
         retweet_count INT,
         retweeted_status STRUCT<
            text:STRING,
            user:STRUCT<screen_name:__STRING,name:STRING>>,
         entities STRUCT<
            urls:ARRAY<STRUCT<expanded___url:STRING>>,
            user_mentions:ARRAY<STRUCT<__screen_name:STRING,name:__STRING>>,
            hashtags:ARRAY<STRUCT<text:__STRING>>>,
         text STRING,
         user STRUCT<
            screen_name:STRING,
            name:STRING,
            friends_count:INT,
            followers_count:INT,
            statuses_count:INT,
            verified:BOOLEAN,
            utc_offset:INT,
            time_zone:STRING>,
         in_reply_to_screen_name STRING
    )
    ROW FORMAT SERDE 'com.cloudera.hive.serde.__JSONSerDe'
    LOCATION '/flume/Tweets_A';


    do I have to add "com.cloudera.hive.serde.__JSONSerDe'" an other time to put data in elasticsearch ?? if yes,How !!

    Thank you so very much


    2014-05-21 10:36 GMT+01:00 Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__>:


         The error indicates that your script refers to a JSonSerDe class (presumably from Cloudera) which cannot be
    found in
         the classpath:

         "> Caused by: java.lang.____ClassNotFoundException: com.cloudera.hive.serde.____JSONSerDe"


         In fact if you read through the stacktrace, you'll notice that this comes up while the job configuration
         is being read and this class name encountered but the class definition itself cannot be resolved.

         I'm not sure what causes this class to be added to the classpath of your script - it could be your script
    or the
         client configuration.
         Note the class is not used or added by ES-hadoop itself.

         I'm not sure what distro you are using (es-hadoop works across Apache Hadoop, Cloudera, HDP, MapR, Pivotal,
    etc...)
         but you could try running some
         basic Hive scripts (without es-hadoop) first to make sure the environment is sane and that you can properly
    read and
         write from HDFS and then add
         es-hadoop into the mix.

         Hope this helps,


         On 5/21/14 12:15 PM, hanine haninne wrote:

             Thank you so much for ur answer
             here is what it gives me now

                 INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;

             Total MapReduce jobs = 1
             Launching Job 1 out of 1
             Number of reduce tasks is set to 0 since there's no reduce operator
             Starting Job = job_201405210900_0002, Tracking URL =
    http://master:50030/____jobdetails.jsp?jobid=job_____201405210900_0002
    <http://master:50030/__jobdetails.jsp?jobid=job___201405210900_0002>
             <http://master:50030/__jobdetails.jsp?jobid=job___201405210900_0002
    <http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002>>
             Kill Command = /home/hduser/hadoop/libexec/..____/bin/hadoop job  -kill job_201405210900_0002

             Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0
             2014-05-21 10:12:05,285 Stage-0 map = 0%,  reduce = 0%
             2014-05-21 10:12:23,408 Stage-0 map = 100%,  reduce = 100%
             Ended Job = job_201405210900_0002 with errors
             Error during job, obtaining debugging information...
             Job Tracking URL: http://master:50030/____jobdetails.jsp?jobid=job_____201405210900_0002
    <http://master:50030/__jobdetails.jsp?jobid=job___201405210900_0002>
             <http://master:50030/__jobdetails.jsp?jobid=job___201405210900_0002
    <http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002>>
             Examining task ID: task_201405210900_0002_m_____000002 (and more) from job job_201405210900_0002


             Task with the most failures(4):
             -----
             Task ID:
                 task_201405210900_0002_m_____000000

             URL:
    http://master:50030/____taskdetails.jsp?jobid=job_____201405210900_0002&tipid=task_____201405210900_0002_m_000000 <http://master:50030/__taskdetails.jsp?jobid=job___201405210900_0002&tipid=task___201405210900_0002_m_000000>


    <http://master:50030/__taskdetails.jsp?jobid=job___201405210900_0002&tipid=task___201405210900_0002_m_000000
    <http://master:50030/taskdetails.jsp?jobid=job_201405210900_0002&tipid=task_201405210900_0002_m_000000>>
             -----
             Diagnostic Messages for this Task:
             java.lang.RuntimeException: Error in configuring object
                   at org.apache.hadoop.util.____ReflectionUtils.setJobConf(____ReflectionUtils.java:93)
                   at org.apache.hadoop.util.____ReflectionUtils.setConf(____ReflectionUtils.java:64)
                   at org.apache.hadoop.util.____ReflectionUtils.newInstance(____ReflectionUtils.java:117)
                   at org.apache.hadoop.mapred.____MapTask.runOldMapper(MapTask.____java:426)
                   at org.apache.hadoop.mapred.____MapTask.run(MapTask.java:366)
                   at org.apache.hadoop.mapred.____Child$4.run(Child.java:255)

                   at java.security.____AccessController.doPrivileged(____Native Method)
                   at javax.security.auth.Subject.____doAs(Subject.java:415)
                   at org.apache.hadoop.security.____UserGroupInformation.doAs(____UserGroupInformation.java:____1190)
                   at org.apache.hadoop.mapred.____Child.main(Child.java:249)
             Caused by: java.lang.reflect.____InvocationTargetException

                   at sun.reflect.____NativeMethodAccessorImpl.____invoke0(Native Method)
                   at sun.reflect.____NativeMethodAccessorImpl.____invoke(____NativeMethodAccessorImpl.java:____57)
                   at
    sun.reflect.____DelegatingMethodAccessorImpl.____invoke(____DelegatingMethodAccessorImpl.____java:43)
                   at java.lang.reflect.Method.____invoke(Method.java:606)
                   at org.apache.hadoop.util.____ReflectionUtils.setJobConf(____ReflectionUtils.java:88)

                   ... 9 more
             Caused by: java.lang.RuntimeException: Error in configuring object
                   at org.apache.hadoop.util.____ReflectionUtils.setJobConf(____ReflectionUtils.java:93)
                   at org.apache.hadoop.util.____ReflectionUtils.setConf(____ReflectionUtils.java:64)
                   at org.apache.hadoop.util.____ReflectionUtils.newInstance(____ReflectionUtils.java:117)
                   at org.apache.hadoop.mapred.____MapRunner.configure(MapRunner.____java:34)
                   ... 14 more
             Caused by: java.lang.reflect.____InvocationTargetException

                   at sun.reflect.____NativeMethodAccessorImpl.____invoke0(Native Method)
                   at sun.reflect.____NativeMethodAccessorImpl.____invoke(____NativeMethodAccessorImpl.java:____57)
                   at
    sun.reflect.____DelegatingMethodAccessorImpl.____invoke(____DelegatingMethodAccessorImpl.____java:43)
                   at java.lang.reflect.Method.____invoke(Method.java:606)
                   at org.apache.hadoop.util.____ReflectionUtils.setJobConf(____ReflectionUtils.java:88)

                   ... 17 more
             Caused by: java.lang.RuntimeException: Map operator initialization failed
                   at org.apache.hadoop.hive.ql.____exec.ExecMapper.configure(____ExecMapper.java:121)
                   ... 22 more
             Caused by: org.apache.hadoop.hive.ql.____metadata.HiveException: java.lang.____ClassNotFoundException:
             com.cloudera.hive.serde.____JSONSerDe
                   at org.apache.hadoop.hive.ql.____exec.MapOperator.____getConvertedOI(MapOperator.____java:463)
                   at org.apache.hadoop.hive.ql.____exec.MapOperator.setChildren(____MapOperator.java:479)
                   at org.apache.hadoop.hive.ql.____exec.ExecMapper.configure(____ExecMapper.java:90)
                   ... 22 more
             Caused by: java.lang.____ClassNotFoundException: com.cloudera.hive.serde.____JSONSerDe
                   at java.net.URLClassLoader$1.run(____URLClassLoader.java:366)
                   at java.net.URLClassLoader$1.run(____URLClassLoader.java:355)
                   at java.security.____AccessController.doPrivileged(____Native Method)
                   at java.net.URLClassLoader.____findClass(URLClassLoader.java:____354)
                   at java.lang.ClassLoader.____loadClass(ClassLoader.java:____424)
                   at sun.misc.Launcher$____AppClassLoader.loadClass(____Launcher.java:308)
                   at java.lang.ClassLoader.____loadClass(ClassLoader.java:____357)
                   at java.lang.Class.forName0(____Native Method)
                   at java.lang.Class.forName(Class.____java:270)
                   at org.apache.hadoop.conf.____Configuration.getClassByName(____Configuration.java:810)
                   at org.apache.hadoop.hive.ql.____exec.MapOperator.____getConvertedOI(MapOperator.____java:422)
                   ... 24 more


             FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.____exec.MapRedTask

             MapReduce Jobs Launched:
             Job 0: Map: 1   HDFS Read: 0 HDFS Write: 0 FAIL
             Total MapReduce CPU Time Spent: 0 msec



             2014-05-20 14:11 GMT+01:00 Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>
             <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>>__>__>:



                  1. You don't need to add the hive jars (hive-serdes) to your script - these are already part of
    the runtime
                  2. use es-hadoop-2.0.RC1, 1.3 M2 is fairly old
                  3. Make sure that the jar is actually available and that is potentially accessible by Hive. Also
    do note that
                  when running Hive against a cluster, it's best to have the jars available on all nodes hence why
    most folks
             copy
                  them to HDFS and refer to that location.
                  While /home/hduser might be locally available, it might not exist on the Hive server - as
    indicated by the
             error.

                  Hope this helps,


                  On 5/20/14 3:58 PM, hanine haninne wrote:


                      Hello everybody

                      I m trying to take data from Hive and put it in elasticsearch
                      here is the script and the error

                      hive> ADD JAR /home/hduser/hadoop/hive-0.11.______0/lib/hive-serdes-1.0-______SNAPSHOT.jar;
                      Added /home/hduser/hadoop/hive-0.11.______0/lib/hive-serdes-1.0-______SNAPSHOT.jar to class path
                      Added resource: /home/hduser/hadoop/hive-0.11.______0/lib/hive-serdes-1.0-______SNAPSHOT.jar


                      hive> CREATE EXTERNAL TABLE tweetsES (
                            >    id BIGINT,
                            >    created_at STRING,
                            >    source STRING,
                            >    favorited BOOLEAN,
                            >    retweet_count INT,
                            >    retweeted_status STRUCT<
                            >       text:STRING,
                            >       user:STRUCT<screen_name:______STRING,name:STRING>>,
                            >    entities STRUCT<
                            >       urls:ARRAY<STRUCT<expanded_______url:STRING>>,
                            >       user_mentions:ARRAY<STRUCT<______screen_name:STRING,name:______STRING>>,
                            >       hashtags:ARRAY<STRUCT<text:______STRING>>>,


                            >    text STRING,
                            >    user STRUCT<
                            >       screen_name:STRING,
                            >       name:STRING,
                            >       friends_count:INT,
                            >       followers_count:INT,
                            >       statuses_count:INT,
                            >       verified:BOOLEAN,
                            >       utc_offset:INT,
                            >       time_zone:STRING>,
                            >    in_reply_to_screen_name STRING
                            > )
                            > STORED BY 'org.elasticsearch.hadoop.______hive.EsStorageHandler'
                            > TBLPROPERTIES('es.resource' = 'reseaux_sociaux/tweets','es.______input.json' = 'yes');


                      OK
                      Time taken: 4.292 seconds
                      hive> ADD JAR /home/hduser/hadoop/______elasticsearch-hadoop-1.3.0.M2.______jar;
                      Added /home/hduser/hadoop/______elasticsearch-hadoop-1.3.0.M2.______jar to class path
                      Added resource: /home/hduser/hadoop/______elasticsearch-hadoop-1.3.0.M2.______jar


                      hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
                      Total MapReduce jobs = 1
                      Launching Job 1 out of 1
                      Number of reduce tasks is set to 0 since there's no reduce operator
                      java.io.FileNotFoundException: File does not exist:
             /home/hduser/hadoop/______elasticsearch-hadoop-1.3.0.M2.______jar
                            at

    org.apache.hadoop.hdfs.______DistributedFileSystem.______getFileStatus(______DistributedFileSystem.java:______558)
                            at
    org.apache.hadoop.filecache.______DistributedCache.______getFileStatus(______DistributedCache.java:185)
                            at


    org.apache.hadoop.filecache.________TrackerDistributedCacheManager______.getFileStatus(______TrackerDistributedCacheManager______.java:723)
                            at


    org.apache.hadoop.filecache.________TrackerDistributedCacheManager______.determineTimestamps(______TrackerDistributedCacheManager______.java:778)
                            at


    org.apache.hadoop.filecache.________TrackerDistributedCacheManager______.______determineTimestampsAndCacheVis______ibilities(______TrackerDistributedCacheManager______.java:755)
                            at
    org.apache.hadoop.mapred.______JobClient.______copyAndConfigureFiles(______JobClient.java:843)
                            at
    org.apache.hadoop.mapred.______JobClient.______copyAndConfigureFiles(______JobClient.java:734)
                            at org.apache.hadoop.mapred.______JobClient.access$400(______JobClient.java:179)
                            at org.apache.hadoop.mapred.______JobClient$2.run(JobClient.______java:951)
                            at org.apache.hadoop.mapred.______JobClient$2.run(JobClient.______java:936)
                            at java.security.______AccessController.doPrivileged(______Native Method)
                            at javax.security.auth.Subject.______doAs(Subject.java:415)
                            at
    org.apache.hadoop.security.______UserGroupInformation.doAs(______UserGroupInformation.java:______1190)
                            at org.apache.hadoop.mapred.______JobClient.submitJobInternal(______JobClient.java:936)
                            at org.apache.hadoop.mapred.______JobClient.submitJob(JobClient.______java:910)
                            at org.apache.hadoop.hive.ql.______exec.ExecDriver.execute(______ExecDriver.java:447)
                            at org.apache.hadoop.hive.ql.______exec.MapRedTask.execute(______MapRedTask.java:138)
                            at org.apache.hadoop.hive.ql.______exec.Task.executeTask(Task.______java:144)
                            at org.apache.hadoop.hive.ql.______exec.TaskRunner.runSequential(______TaskRunner.java:57)
                            at org.apache.hadoop.hive.ql.______Driver.launchTask(Driver.java:______1355)
                            at org.apache.hadoop.hive.ql.______Driver.execute(Driver.java:______1139)
                            at org.apache.hadoop.hive.ql.______Driver.run(Driver.java:945)
                            at org.apache.hadoop.hive.cli.______CliDriver.processLocalCmd(______CliDriver.java:259)
                            at org.apache.hadoop.hive.cli.______CliDriver.processCmd(______CliDriver.java:216)
                            at org.apache.hadoop.hive.cli.______CliDriver.processLine(______CliDriver.java:413)
                            at org.apache.hadoop.hive.cli.______CliDriver.run(CliDriver.java:______756)
                            at org.apache.hadoop.hive.cli.______CliDriver.main(CliDriver.__java:____614)
                            at sun.reflect.______NativeMethodAccessorImpl.______invoke0(Native Method)
                            at
    sun.reflect.______NativeMethodAccessorImpl.______invoke(______NativeMethodAccessorImpl.java:______57)
                            at

    sun.reflect.______DelegatingMethodAccessorImpl.______invoke(______DelegatingMethodAccessorImpl.______java:43)
                            at java.lang.reflect.Method.______invoke(Method.java:606)
                            at org.apache.hadoop.util.RunJar.______main(RunJar.java:160)

                      Job Submission failed with exception 'java.io <http://java.io> <http://java.io>
             <http://java.io>.______FileNotFoundException(File does not exist:
                      /home/hduser/hadoop/______elasticsearch-hadoop-1.3.0.M2.______jar)'
                      FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.______exec.MapRedTask


                      That path it exist :/home/hduser/hadoop/______elasticsearch-hadoop-1.3.0.M2.______jar


                      Any help would be so appreciable
                      Thanks

                      --
                      You received this message because you are subscribed to the Google Groups "elasticsearch" group.
                      To unsubscribe from this group and stop receiving emails from it, send an email to
                      elasticsearch+unsubscribe@__go____oglegroups.com <http://go__oglegroups.com>
    <http://googlegroups.com>
             <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>
                      <mailto:elasticsearch+______unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B____unsubscribe@googlegroups.com>
             <mailto:elasticsearch%2B____unsubscribe@googlegroups.com
    <mailto:elasticsearch%252B__unsubscribe@googlegroups.com>>
    <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>>.



                      To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__d1613b48-____e58d-4077-a579-__92b46a97bcff%______40googlegroups.com
    <https://groups.google.com/d/____msgid/elasticsearch/d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com>

    <https://groups.google.com/d/____msgid/elasticsearch/d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-__e58d-4077-a579-92b46a97bcff%__40googlegroups.com>>


    <https://groups.google.com/d/____msgid/elasticsearch/d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-__e58d-4077-a579-92b46a97bcff%__40googlegroups.com>

    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-__e58d-4077-a579-92b46a97bcff%__40googlegroups.com
    <https://groups.google.com/d/msgid/elasticsearch/d1613b48-e58d-4077-a579-92b46a97bcff%40googlegroups.com>>>


    <https://groups.google.com/d/______msgid/elasticsearch/__d1613b48-____e58d-4077-a579-__92b46a97bcff%______40googlegroups.com?utm_____medium=__email&utm_source=____footer
    <https://groups.google.com/d/____msgid/elasticsearch/d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com?utm___medium=__email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-__e58d-4077-a579-92b46a97bcff%__40googlegroups.com?utm_medium=__email&utm_source=footer>>



    <https://groups.google.com/d/____msgid/elasticsearch/d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-__e58d-4077-a579-92b46a97bcff%__40googlegroups.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-__e58d-4077-a579-92b46a97bcff%__40googlegroups.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/d1613b48-e58d-4077-a579-92b46a97bcff%40googlegroups.com?utm_medium=email&utm_source=footer>>>>.

                      For more options, visit https://groups.google.com/d/______optout
    <https://groups.google.com/d/____optout> <https://groups.google.com/d/____optout
    <https://groups.google.com/d/__optout>>

             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>.



                  --
                  Costin


                  --
                  You received this message because you are subscribed to the Google Groups "elasticsearch" group.
                  To unsubscribe from this group and stop receiving emails from it, send an email to
                  elasticsearch+unsubscribe@__go____oglegroups.com <http://go__oglegroups.com> <http://googlegroups.com>
             <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>.


                  To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__537B549F.____8020501%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/537B549F.____8020501%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/537B549F.____8020501%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/537B549F.__8020501%40gmail.com>>

                  <https://groups.google.com/d/____msgid/elasticsearch/537B549F.____8020501%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/537B549F.__8020501%40gmail.com>
             <https://groups.google.com/d/__msgid/elasticsearch/537B549F.__8020501%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/537B549F.8020501%40gmail.com>>>.

                  For more options, visit https://groups.google.com/d/______optout
    <https://groups.google.com/d/____optout> <https://groups.google.com/d/____optout
    <https://groups.google.com/d/__optout>>

             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>.



             --
             You received this message because you are subscribed to the Google Groups "elasticsearch" group.
             To unsubscribe from this group and stop receiving emails from it, send an email to
             elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
    <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>
             <mailto:elasticsearch+____unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B__unsubscribe@googlegroups.com> <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>>.
             To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/CANXJSR9-____xpGoJR4gvPW%3DKKP%3DVr_____5MxyDoUJKtbpqM6CsYs1G4Q%____40mail.gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-__xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%__40mail.gmail.com>

    <https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-__xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%__40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%40mail.gmail.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/CANXJSR9-____xpGoJR4gvPW%3DKKP%3DVr_____5MxyDoUJKtbpqM6CsYs1G4Q%____40mail.gmail.com?utm_medium=____email&utm_source=footer
    <https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-__xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%__40mail.gmail.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-__xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%__40mail.gmail.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%40mail.gmail.com?utm_medium=email&utm_source=footer>>>.

             For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.


         --
         Costin

         --
         You received this message because you are subscribed to the Google Groups "elasticsearch" group.
         To unsubscribe from this group and stop receiving emails from it, send an email to
         elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
    <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>.
         To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/537C738E.____9030001%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/537C738E.__9030001%40gmail.com>
         <https://groups.google.com/d/__msgid/elasticsearch/537C738E.__9030001%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/537C738E.9030001%40gmail.com>>.


         For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.


    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>>.
    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/__CANXJSR8Gy4qhM8Lzm5BHCr2-__xptLKBSdm00nCrz-JtnW59jWcw%__40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANXJSR8Gy4qhM8Lzm5BHCr2-xptLKBSdm00nCrz-JtnW59jWcw%40mail.gmail.com>
    <https://groups.google.com/d/__msgid/elasticsearch/__CANXJSR8Gy4qhM8Lzm5BHCr2-__xptLKBSdm00nCrz-JtnW59jWcw%__40mail.gmail.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CANXJSR8Gy4qhM8Lzm5BHCr2-xptLKBSdm00nCrz-JtnW59jWcw%40mail.gmail.com?utm_medium=email&utm_source=footer>>.

    For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.


--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/537C8751.__1030908%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/537C8751.1030908%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANXJSR_pj6i55jwA3wYiGgh1d_q86DpcNRKLZzhk-V97vWFLaQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANXJSR_pj6i55jwA3wYiGgh1d_q86DpcNRKLZzhk-V97vWFLaQ%40mail.gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/537E0B0B.3070307%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


(hanine) #9

Hello,
I m sorry,but it did not work yet. I thinking I m missing something,Here is
what I did from the beginning ( from getting tweets with flume )

1-script flume (Tweets.conf)
2-store data in HDFS as indicated in the script
3-then I create a table with hive and get data there (in order to have the
possibility to communicate with elasticsearch after) the scripte hive is in
hive_tweets
now I wanna put that data in elasticsearch(head) in order to show it in an
ergonomic interface (like image1 )
so that I tried this script (hiveES)
when I execute it, it gives me :

> STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
> TBLPROPERTIES('es.resource' =

'reseaux_sociaux/tweets','es.input.json' = 'yes');
OK
Time taken: 0.133 seconds
hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201405272117_0002, Tracking URL =
http://master:50030/jobdetails.jsp?jobid=job_201405272117_0002
Kill Command = /home/hduser/hadoop/libexec/../bin/hadoop job -kill
job_201405272117_0002
Hadoop job information for Stage-0: number of mappers: 1; number of
reducers: 0
2014-05-27 21:51:14,893 Stage-0 map = 0%, reduce = 0%
2014-05-27 21:51:35,019 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201405272117_0002 with errors
Error during job, obtaining debugging information...
Job Tracking URL:
http://master:50030/jobdetails.jsp?jobid=job_201405272117_0002
Examining task ID: task_201405272117_0002_m_000002 (and more) from job
job_201405272117_0002

Task with the most failures(4):

Task ID:
task_201405272117_0002_m_000000

URL:

http://master:50030/taskdetails.jsp?jobid=job_201405272117_0002&tipid=task_201405272117_0002_m_000000

Diagnostic Messages for this Task:
java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {"id":467649114269028352,"created_at":"Sat May 17 12:53:32
+0000 2014","source":"<a href="http://www.apple.com"
rel="nofollow">iOS","favorited":false,"retweet_count":0,"retweeted_status":null,"entities":{"urls":[{"expanded_url":"
https://www.google.co.uk/search?q=viera+lifting+fa+cup+for+arsenal&client=safari&hl=en-gb&source=lnms&tbm=isch&sa=X&ei=alp3U4LrK82UOufRgaAK&ved=0CAYQ_AUoAQ&biw=320&bih=460#facrc=&imgrc=-QK5IhwQT8AS8M%253A%3B2yfUWG7-cngS-M%3Bhttp%253A%252F%252Ffarm4.static.flickr.com%252F3777%252F10838930556_618b87d7cc_m.jpg%3Bhttp%253A%252F%252Fwhotalking.com%252Fflickr%252F2-0%252BChelsea%3B1024%3B669"}],"user_mentions":[{"screen_name":"jameswilliams46","name":"James
Williams"}],"hashtags":[]},"text":"@jameswilliams46
https://t.co/CAmxdagAym","user":{"screen_name":"SSimpson7379","name":"Stephen
Simpson","friends_count":258,"followers_count":266,"statuses_count":2609,"verified":false,"utc_offset":3600,"time_zone":"London"},"in_reply_to_screen_name":"jameswilliams46"}
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:162)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row {"id":467649114269028352,"created_at":"Sat May
17 12:53:32 +0000 2014","source":"<a href="http://www.apple.com"
rel="nofollow">iOS","favorited":false,"retweet_count":0,"retweeted_status":null,"entities":{"urls":[{"expanded_url":"
https://www.google.co.uk/search?q=viera+lifting+fa+cup+for+arsenal&client=safari&hl=en-gb&source=lnms&tbm=isch&sa=X&ei=alp3U4LrK82UOufRgaAK&ved=0CAYQ_AUoAQ&biw=320&bih=460#facrc=
&imgrc=-QK5IhwQT8AS8M%253A%3B2yfUWG7-cngS-M%3Bhttp%253A%252F%252Ffarm4.static.flickr.com%252F3777%252F10838930556_618b87d7cc_m.jpg%3Bhttp%253A%252F%252Fwhotalking.com%252Fflickr%252F2-0%252BChelsea%3B1024%3B669"}],"user_mentions":[{"screen_name":"jameswilliams46","name":"James
Williams"}],"hashtags":[]},"text":"@jameswilliams46
https://t.co/CAmxdagAym","user":{"screen_name":"SSimpson7379","name":"Stephen
Simpson","friends_count":258,"followers_count":266,"statuses_count":2609,"verified":false,"utc_offset":3600,"time_zone":"London"},"in_reply_to_screen_name":"jameswilliams46"}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:671)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144)
... 8 more
Caused by: java.lang.IllegalArgumentException: When using JSON input, only
one field is expected
at org.elasticsearch.hadoop.util.Assert.isTrue(Assert.java:48)
at
org.elasticsearch.hadoop.hive.HiveBytesConverter.convert(HiveBytesConverter.java:48)
at
org.elasticsearch.hadoop.serialization.command.JsonTemplatedCommand.preProcess(JsonTemplatedCommand.java:56)
at
org.elasticsearch.hadoop.serialization.command.TemplatedCommand.write(TemplatedCommand.java:73)
at org.elasticsearch.hadoop.hive.EsSerDe.serialize(EsSerDe.java:135)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:617)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:652)
... 9 more

FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

Any help would be so appreciable
Thank you so much

2014-05-22 15:34 GMT+01:00 Costin Leau costin.leau@gmail.com:

Hi,

It looks like you have two tables - one that uses the JSONSerDe from
cloudera and another one using es-hadoop.
You configured your es-hadoop table to consider the input as json however
it does not receive the proper format (as the exception indicates).
See this [1] section of the documentation for more information.

Let me restate again that you do not the data to Elasticsearch and
es-hadoop to be in JSON format - unless it actually is in JSON
(as in JSON files are stored in HDFS) simply send whatever data you have
in Hive to es-hadoop and it will do the JSON translation automatically
and typically much more efficient.
Again the docs [2] discuss this in detail. Frankly, I'm not sure why you
want to use the JSONSerDe.

To conclude: if the data is in raw format in HDFS, just feed to es-hadoop
with the input configured as JSON, if it's in a different format,
load it in Hive as you would typically do and then insert it into the
es-hadoop table.

Hope this helps,

[1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/
current/hive.html#writing-json-hive
[2] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/
current/hive.html#hive

On 5/22/14 12:13 PM, hanine haninne wrote:

Hello again,
I m trying to get some tweets from the HDFS and put it in
elasticsearch(head) in order to make search on it, I did what
u said and I m so thankful, But it start reading and does not continue. I
don't know why ?!!
here is what I did :

hive -hiveconf hive.aux.jars.path=/ES_Hadoop/
dist/elasticsearch-hadoop-2.0.0.RC1.jar

Logging initialized using configuration in
jar:file:/home/hduser/hadoop/hive-0.11.0/lib/hive-common-0.
11.0.jar!/hive-log4j.properties
Hive history file=/tmp/hduser/hive_job_log_hduser_8166@mouna-pc_
201405220958_2036150447.txt

hive> ADD JAR /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-
SNAPSHOT.jar;
Added /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar
to class path
Added resource: /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-
SNAPSHOT.jar
hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201405220937_0005, Tracking URL = http://master:50030/
jobdetails.jsp?jobid=job_201405220937_0005
Kill Command = /home/hduser/hadoop/libexec/../bin/hadoop job -kill
job_201405220937_0005
Hadoop job information for Stage-0: number of mappers: 1; number of
reducers: 0
2014-05-22 10:01:39,184 Stage-0 map = 0%, reduce = 0%
2014-05-22 10:01:59,320 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201405220937_0005 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://master:50030/jobdetails.jsp?jobid=job_
201405220937_0005
Examining task ID: task_201405220937_0005_m_000002 (and more) from job
job_201405220937_0005

Task with the most failures(4):

Task ID:
task_201405220937_0005_m_000000

URL:
http://master:50030/taskdetails.jsp?jobid=job_
201405220937_0005&tipid=task_201405220937_0005_m_000000

Diagnostic Messages for this Task:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row
{"id":467649114269028352,"created_at":"Sat May 17 12:53:32 +0000
2014","source":"<a href="http://www.apple.com"
rel="nofollow">iOS","favorited":false,"retweet_
count":0,"retweeted_status":null,"entities":{"urls":[{"expanded_url":"
https://www.google.co.uk/search?q=viera+lifting+fa+cup+for+arsenal&
client=safari&hl=en-gb&source=lnms&tbm=isch&sa=X&ei=
alp3U4LrK82UOufRgaAK&ved=0CAYQ_AUoAQ&biw=320&bih=460#
facrc=&imgrc=-QK5IhwQT8AS8M%253A%3B2yfUWG7-cngS-M%3Bhttp%
253A%252F%252Ffarm4.static.flickr.com%252F3777%
252F10838930556_618b87d7cc_m.jpg%3Bhttp%253A%252F%
252Fwhotalking.com%252Fflickr%252F2-0%252BChelsea%3B1024%3B669
"}],"user_mentions":[{"screen_name":"jameswilliams46","name":"James
Williams"}],"hashtags":[]},"text":"@jameswilliams46
https://t.co/CAmxdagAym","user":{"screen_name":"
SSimpson7379","name":"Stephen
Simpson","friends_count":258,"followers_count":266,"
statuses_count":2609,"verified":false,"utc_offset":
3600,"time_zone":"London"},"in_reply_to_screen_name":"jameswilliams46"}
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(
ExecMapper.java:162)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(
UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row
{"id":467649114269028352,"created_at":"Sat May 17 12:53:32 +0000
2014","source":"<a href="http://www.apple.com"
rel="nofollow">iOS","favorited":false,"retweet

count":0,"retweeted_status":null,"entities":{"urls":[{"expanded_url":"
https://www.google.co.uk/search?q=viera+lifting+fa+cup+for+arsenal&
client=safari&hl=en-gb&source=lnms&tbm=isch&sa=X&ei=
alp3U4LrK82UOufRgaAK&ved=0CAYQ_AUoAQ&biw=320&bih=460#
facrc=_&imgrc=-QK5IhwQT8AS8M%253A%3B2yfUWG7-cngS-M%3Bhttp%
253A%252F%252Ffarm4.static.flickr.com%252F3777%
252F10838930556_618b87d7cc_m.jpg%3Bhttp%253A%252F%
252Fwhotalking.com%252Fflickr%252F2-0%252BChelsea%3B1024%3B669
"}],"user_mentions":[{"screen_name":"jameswilliams46","name":"James
Williams"}],"hashtags":[]},"text":"@jameswilliams46
https://t.co/CAmxdagAym","user":{"screen_name":"
SSimpson7379","name":"Stephen
Simpson","friends_count":258,"followers_count":266,"
statuses_count":2609,"verified":false,"utc_offset":
3600,"time_zone":"London"},"in_reply_to_screen_name":"jameswilliams46"}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(
MapOperator.java:671)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(
ExecMapper.java:144)
... 8 more
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException:
When using JSON input, only one field is expected
at org.elasticsearch.hadoop.util.Assert.isTrue(Assert.java:50)
at org.elasticsearch.hadoop.hive.HiveBytesConverter.convert(
HiveBytesConverter.java:48)
at org.elasticsearch.hadoop.serialization.bulk.
JsonTemplatedBulk.preProcess(JsonTemplatedBulk.java:58)
at org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.write(
TemplatedBulk.java:75)
at org.elasticsearch.hadoop.hive.EsSerDe.serialize(EsSerDe.java:128)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.
processOp(FileSinkOperator.java:617)
at org.apache.hadoop.hive.ql.exec.Operator.process(
Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(
Operator.java:832)
at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(
SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(
Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(
Operator.java:832)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.
processOp(TableScanOperator.java:90)
at org.apache.hadoop.hive.ql.exec.Operator.process(
Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(
Operator.java:832)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(
MapOperator.java:652)
... 9 more

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.
exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

when I did not add that jar "ADD JAR /home/hduser/hadoop/hive-0.11.
0/lib/hive-serdes-1.0-SNAPSHOT.jar;" it gave me that
error "Caused by: java.lang.ClassNotFoundException:
com.cloudera.hive.serde.JSONSerDe
"

Thank you so very much for your helping

2014-05-21 12:00 GMT+01:00 Costin Leau <costin.leau@gmail.com <mailto:
costin.leau@gmail.com>>:

If you plan to use es-hadoop, there's no need to do any JSON

conversion (es-hadoop does that automatically) - see
the documentation at [1]
for more information.
If not than I recommend reaching out to the providers of the package
for more information on its configuration.

[1] elasticsearch.org/hadoop <http://elasticsearch.org/hadoop>



On 5/21/14 1:17 PM, hanine haninne wrote:

    yes, you are right.Thanks so much

    well I tried hive with HDFS and it works very well,by the way I m

using hive-0.11.0 , hadoop-1.2.1 and
elasticsearch-1.0.1
Here is the script of the tables
CREATE EXTERNAL TABLE tweets (
id BIGINT,
created_at STRING,
source STRING,
favorited BOOLEAN,
retweet_count INT,
retweeted_status STRUCT<
text:STRING,
user:STRUCT<screen_name:__STRING,name:STRING>>,
entities STRUCT<
urls:ARRAY<STRUCT<expanded___url:STRING>>,
user_mentions:ARRAY<STRUCT<screen_name:STRING,name:
STRING>>,
hashtags:ARRAY<STRUCTtext:__STRING>>,

         text STRING,
         user STRUCT<
            screen_name:STRING,
            name:STRING,
            friends_count:INT,
            followers_count:INT,
            statuses_count:INT,
            verified:BOOLEAN,
            utc_offset:INT,
            time_zone:STRING>,
         in_reply_to_screen_name STRING
    )
    ROW FORMAT SERDE 'com.cloudera.hive.serde.__JSONSerDe'
    LOCATION '/flume/Tweets_A';


    do I have to add "com.cloudera.hive.serde.__JSONSerDe'" an other

time to put data in elasticsearch ?? if yes,How !!

    Thank you so very much


    2014-05-21 10:36 GMT+01:00 Costin Leau <costin.leau@gmail.com<mailto:

costin.leau@gmail.com>
<mailto:costin.leau@gmail.com mailto:costin.leau@gmail.com>__>:

         The error indicates that your script refers to a JSonSerDe

class (presumably from Cloudera) which cannot be
found in
the classpath:

         "> Caused by: java.lang.____ClassNotFoundException:

com.cloudera.hive.serde.____JSONSerDe"

         In fact if you read through the stacktrace, you'll notice

that this comes up while the job configuration
is being read and this class name encountered but the class
definition itself cannot be resolved.

         I'm not sure what causes this class to be added to the

classpath of your script - it could be your script
or the
client configuration.
Note the class is not used or added by ES-hadoop itself.

         I'm not sure what distro you are using (es-hadoop works

across Apache Hadoop, Cloudera, HDP, MapR, Pivotal,
etc...)
but you could try running some
basic Hive scripts (without es-hadoop) first to make sure
the environment is sane and that you can properly
read and
write from HDFS and then add
es-hadoop into the mix.

         Hope this helps,


         On 5/21/14 12:15 PM, hanine haninne wrote:

             Thank you so much for ur answer
             here is what it gives me now

                 INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;

             Total MapReduce jobs = 1
             Launching Job 1 out of 1
             Number of reduce tasks is set to 0 since there's no

reduce operator
Starting Job = job_201405210900_0002, Tracking URL =
http://master:50030/jobdetails.jsp?jobid=job_
201405210900_0002

    <http://master:50030/__jobdetails.jsp?jobid=job___

201405210900_0002>
<http://master:50030/jobdetails.jsp?jobid=job_
201405210900_0002
http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002>
Kill Command = /home/hduser/hadoop/libexec/..____/bin/hadoop
job -kill job_201405210900_0002

             Hadoop job information for Stage-0: number of mappers:

1; number of reducers: 0
2014-05-21 10:12:05,285 Stage-0 map = 0%, reduce = 0%
2014-05-21 10:12:23,408 Stage-0 map = 100%, reduce =
100%
Ended Job = job_201405210900_0002 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://master:50030/____
jobdetails.jsp?jobid=job_____201405210900_0002
<http://master:50030/jobdetails.jsp?jobid=job_
201405210900_0002>

             <http://master:50030/__jobdetails.jsp?jobid=job___

201405210900_0002
http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002>
Examining task ID: task_201405210900_0002_m_____000002
(and more) from job job_201405210900_0002

             Task with the most failures(4):
             -----
             Task ID:
                 task_201405210900_0002_m_____000000

             URL:
    http://master:50030/____taskdetails.jsp?jobid=job_____

201405210900_0002&tipid=task_____201405210900_0002_m_000000 <
http://master:50030/taskdetails.jsp?jobid=job_
201405210900_0002&tipid=task___201405210900_0002_m_000000>

    <http://master:50030/__taskdetails.jsp?jobid=job___

201405210900_0002&tipid=task___201405210900_0002_m_000000
<http://master:50030/taskdetails.jsp?jobid=job_
201405210900_0002&tipid=task_201405210900_0002_m_000000>>
-----
Diagnostic Messages for this Task:
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.____
ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.

ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.

ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.

MapTask.runOldMapper(MapTask.java:426)
at org.apache.hadoop.mapred.

MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.____
Child$4.run(Child.java:255)

                   at java.security.____

AccessController.doPrivileged(_Native Method)
at javax.security.auth.Subject.

doAs(Subject.java:415)
at org.apache.hadoop.security.

_UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.

Child.main(Child.java:249)
Caused by: java.lang.reflect.

InvocationTargetException

                   at sun.reflect.____NativeMethodAccessorImpl.____invoke0(Native

Method)
at sun.reflect.NativeMethodAccessorImpl.
invoke(____NativeMethodAccessorImpl.java:____57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.

invoke(Method.java:606)
at org.apache.hadoop.util.

ReflectionUtils.setJobConf(____ReflectionUtils.java:88)

                   ... 9 more
             Caused by: java.lang.RuntimeException: Error in

configuring object
at org.apache.hadoop.util.____
ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.

ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.

ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.

MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.

InvocationTargetException

                   at sun.reflect.____NativeMethodAccessorImpl.____invoke0(Native

Method)
at sun.reflect.NativeMethodAccessorImpl.
invoke(____NativeMethodAccessorImpl.java:____57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.

invoke(Method.java:606)
at org.apache.hadoop.util.

ReflectionUtils.setJobConf(____ReflectionUtils.java:88)

                   ... 17 more
             Caused by: java.lang.RuntimeException: Map operator

initialization failed
at org.apache.hadoop.hive.ql.____
exec.ExecMapper.configure(____ExecMapper.java:121)
... 22 more
Caused by: org.apache.hadoop.hive.ql.____metadata.HiveException:
java.lang.____ClassNotFoundException:
com.cloudera.hive.serde.JSONSerDe
at org.apache.hadoop.hive.ql.

exec.MapOperator.____getConvertedOI(MapOperator.java:463)
at org.apache.hadoop.hive.ql.

exec.MapOperator.setChildren(MapOperator.java:479)
at org.apache.hadoop.hive.ql.

exec.ExecMapper.configure(____ExecMapper.java:90)
... 22 more
Caused by: java.lang.____ClassNotFoundException:
com.cloudera.hive.serde.____JSONSerDe
at java.net.URLClassLoader$1.run(
____URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(
URLClassLoader.java:355)
at java.security.

AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.

findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.

loadClass(ClassLoader.java:424)
at sun.misc.Launcher$

AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.

loadClass(ClassLoader.java:____357)
at java.lang.Class.forName0(____Native Method)
at java.lang.Class.forName(Class.java:270)
at org.apache.hadoop.conf.

Configuration.getClassByName(Configuration.java:810)
at org.apache.hadoop.hive.ql.

exec.MapOperator.____getConvertedOI(MapOperator.____java:422)
... 24 more

             FAILED: Execution Error, return code 2 from

org.apache.hadoop.hive.ql.____exec.MapRedTask

             MapReduce Jobs Launched:
             Job 0: Map: 1   HDFS Read: 0 HDFS Write: 0 FAIL
             Total MapReduce CPU Time Spent: 0 msec



             2014-05-20 14:11 GMT+01:00 Costin Leau <

costin.leau@gmail.com mailto:costin.leau@gmail.com
<mailto:costin.leau@gmail.com mailto:costin.leau@gmail.com>
<mailto:costin.leau@gmail.com <mailto:
costin.leau@gmail.com> <mailto:costin.leau@gmail.com
mailto:costin.leau@gmail.com>>>:

                  1. You don't need to add the hive jars

(hive-serdes) to your script - these are already part of
the runtime
2. use es-hadoop-2.0.RC1, 1.3 M2 is fairly old
3. Make sure that the jar is actually available and
that is potentially accessible by Hive. Also
do note that
when running Hive against a cluster, it's best to
have the jars available on all nodes hence why
most folks
copy
them to HDFS and refer to that location.
While /home/hduser might be locally available, it
might not exist on the Hive server - as
indicated by the
error.

                  Hope this helps,


                  On 5/20/14 3:58 PM, hanine haninne wrote:


                      Hello everybody

                      I m trying to take data from Hive and put it in

elasticsearch
here is the script and the error

                      hive> ADD JAR /home/hduser/hadoop/hive-0.11.

______0/lib/hive-serdes-1.0-______SNAPSHOT.jar;
Added /home/hduser/hadoop/hive-0.11.
______0/lib/hive-serdes-1.0-______SNAPSHOT.jar to class path
Added resource: /home/hduser/hadoop/hive-0.11.
______0/lib/hive-serdes-1.0-______SNAPSHOT.jar

                      hive> CREATE EXTERNAL TABLE tweetsES (
                            >    id BIGINT,
                            >    created_at STRING,
                            >    source STRING,
                            >    favorited BOOLEAN,
                            >    retweet_count INT,
                            >    retweeted_status STRUCT<
                            >       text:STRING,
                            >       user:STRUCT<screen_name:______

STRING,name:STRING>>,
> entities STRUCT<
> urls:ARRAY<STRUCT<expanded____
url:STRING>>,
> user_mentions:ARRAY<STRUCT<

___screen_name:STRING,name:___STRING>>,
> hashtags:ARRAY<STRUCT<text:

___STRING>>>,

                            >    text STRING,
                            >    user STRUCT<
                            >       screen_name:STRING,
                            >       name:STRING,
                            >       friends_count:INT,
                            >       followers_count:INT,
                            >       statuses_count:INT,
                            >       verified:BOOLEAN,
                            >       utc_offset:INT,
                            >       time_zone:STRING>,
                            >    in_reply_to_screen_name STRING
                            > )
                            > STORED BY 'org.elasticsearch.hadoop.____

__hive.EsStorageHandler'
> TBLPROPERTIES('es.resource' =
'reseaux_sociaux/tweets','es.______input.json' = 'yes');

                      OK
                      Time taken: 4.292 seconds
                      hive> ADD JAR /home/hduser/hadoop/______

elasticsearch-hadoop-1.3.0.M2.jar;
Added /home/hduser/hadoop/

elasticsearch-hadoop-1.3.0.M2.jar to class path
Added resource: /home/hduser/hadoop/

elasticsearch-hadoop-1.3.0.M2.______jar

                      hive> INSERT OVERWRITE TABLE tweetsES SELECT *

FROM tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since
there's no reduce operator
java.io.FileNotFoundException: File does not
exist:
/home/hduser/hadoop/______elasticsearch-hadoop-1.3.0.M2.
______jar
at

    org.apache.hadoop.hdfs.______DistributedFileSystem.______

getFileStatus(______DistributedFileSystem.java:______558)
at
org.apache.hadoop.filecache.DistributedCache.
getFileStatus(______DistributedCache.java:185)
at

    org.apache.hadoop.filecache.________

TrackerDistributedCacheManager______.getFileStatus(______
TrackerDistributedCacheManager______.java:723)
at

    org.apache.hadoop.filecache.________

TrackerDistributedCacheManager______.determineTimestamps(______
TrackerDistributedCacheManager______.java:778)
at

    org.apache.hadoop.filecache.________

TrackerDistributedCacheManager______.______determineTimestampsAndCacheVis
______ibilities(TrackerDistributedCacheManager.java:755)
at
org.apache.hadoop.mapred.JobClient.
copyAndConfigureFiles(______JobClient.java:843)
at
org.apache.hadoop.mapred.JobClient.
copyAndConfigureFiles(_JobClient.java:734)
at org.apache.hadoop.mapred.

_JobClient.access$400(_JobClient.java:179)
at org.apache.hadoop.mapred.

_JobClient$2.run(JobClient._java:951)
at org.apache.hadoop.mapred.

_JobClient$2.run(JobClient.java:936)
at java.security.

AccessController.doPrivileged(____Native Method)
at javax.security.auth.Subject.

____doAs(Subject.java:415)
at
org.apache.hadoop.security._____UserGroupInformation.doAs(
_____UserGroupInformation.java:_1190)
at org.apache.hadoop.mapred.

_JobClient.submitJobInternal(_JobClient.java:936)
at org.apache.hadoop.mapred.

_JobClient.submitJob(JobClient.__java:910)
at org.apache.hadoop.hive.ql.

__exec.ExecDriver.execute(__ExecDriver.java:447)
at org.apache.hadoop.hive.ql.

__exec.MapRedTask.execute(__MapRedTask.java:138)
at org.apache.hadoop.hive.ql.

__exec.Task.executeTask(Task.__java:144)
at org.apache.hadoop.hive.ql.

__exec.TaskRunner.runSequential(__TaskRunner.java:57)
at org.apache.hadoop.hive.ql.

__Driver.launchTask(Driver.java:__1355)
at org.apache.hadoop.hive.ql.

__Driver.execute(Driver.java:_1139)
at org.apache.hadoop.hive.ql.

Driver.run(Driver.java:945)
at org.apache.hadoop.hive.cli.

___CliDriver.processLocalCmd(___CliDriver.java:259)
at org.apache.hadoop.hive.cli.

___CliDriver.processCmd(___CliDriver.java:216)
at org.apache.hadoop.hive.cli.

___CliDriver.processLine(___CliDriver.java:413)
at org.apache.hadoop.hive.cli.

___CliDriver.run(CliDriver.java:___756)
at org.apache.hadoop.hive.cli.

___CliDriver.main(CliDriver.java:614)
at sun.reflect.

NativeMethodAccessorImpl.______invoke0(Native Method)
at
sun.reflect.______NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:______57)
at

    sun.reflect.______DelegatingMethodAccessorImpl._

_____invoke(______DelegatingMethodAccessorImpl._java:43)
at java.lang.reflect.Method.

_invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.
______main(RunJar.java:160)

                      Job Submission failed with exception 'java.io <

http://java.io> http://java.io
http://java.io.FileNotFoundException(File does
not exist:
/home/hduser/hadoop/

elasticsearch-hadoop-1.3.0.M2.______jar)'
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.______exec.MapRedTask

                      That path it exist :/home/hduser/hadoop/______

elasticsearch-hadoop-1.3.0.M2.______jar

                      Any help would be so appreciable
                      Thanks

                      --
                      You received this message because you are

subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop
receiving emails from it, send an email to
elasticsearch+unsubscribe@_go
____oglegroups.com http://go__oglegroups.com
http://googlegroups.com
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
<mailto:elasticsearch%252Bunsubscribe@googlegroups.com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>
>
<mailto:elasticsearch+

unsubscribe@googlegroups.com
mailto:elasticsearch%2B____unsubscribe@googlegroups.com
<mailto:elasticsearch%2B____unsubscribe@googlegroups.com
mailto:elasticsearch%2B__unsubscribe@googlegroups.com>
<mailto:elasticsearch%____2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%25__2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch%252Bunsubscribe@googlegroups.
_com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>
>>.

                      To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__

d1613b48-____e58d-4077-a579-__92b46a97bcff%______40googlegroups.com
<https://groups.google.com/d/____msgid/elasticsearch/
d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com>

    <https://groups.google.com/d/____msgid/elasticsearch/

d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/d1613b48-
__e58d-4077-a579-92b46a97bcff%__40googlegroups.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/

d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/d1613b48-
__e58d-4077-a579-92b46a97bcff%__40googlegroups.com>

    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-

__e58d-4077-a579-92b46a97bcff%__40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/d1613b48-
e58d-4077-a579-92b46a97bcff%40googlegroups.com>>>

    <https://groups.google.com/d/______msgid/elasticsearch/__

d1613b48-e58d-4077-a579-92b46a97bcff%
40googlegroups.com?utm_____medium=__email&utm_source=____footer
<https://groups.google.com/d/____msgid/elasticsearch/
d1613b48-____e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/

d1613b48-____e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/d1613b48-
__e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=footer>>

    <https://groups.google.com/d/____msgid/elasticsearch/

d1613b48-____e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/d1613b48-
__e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-

__e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/d1613b48-
e58d-4077-a579-92b46a97bcff%40googlegroups.com?utm_medium=
email&utm_source=footer>>>>.

                      For more options, visit

https://groups.google.com/d/______optout
https://groups.google.com/d/____optout <
https://groups.google.com/d/____optout

    <https://groups.google.com/d/__optout>>

             <https://groups.google.com/d/____optout <

https://groups.google.com/d/__optout>
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>>.

                  --
                  Costin


                  --
                  You received this message because you are

subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving
emails from it, send an email to
elasticsearch+unsubscribe@__go____oglegroups.com <
http://go__oglegroups.com> http://googlegroups.com
<mailto:elasticsearch%____2Bunsubscribe@googlegroups.com
mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
<mailto:elasticsearch%__252Bunsubscribe@googlegroups.com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>
>.

                  To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__

537B549F.____8020501%40gmail.__com
<https://groups.google.com/d/____msgid/elasticsearch/
537B549F.____8020501%40gmail.com>

             <https://groups.google.com/d/____msgid/elasticsearch/

537B549F.____8020501%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/537B549F.
__8020501%40gmail.com>>

                  <https://groups.google.com/d/_

___msgid/elasticsearch/537B549F.____8020501%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/537B549F.
_8020501%40gmail.com>
<https://groups.google.com/d/

_msgid/elasticsearch/537B549F.__8020501%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/537B549F.
8020501%40gmail.com>>>.

                  For more options, visit

https://groups.google.com/d/______optout
https://groups.google.com/d/____optout <
https://groups.google.com/d/____optout
https://groups.google.com/d/__optout>

             <https://groups.google.com/d/____optout <

https://groups.google.com/d/__optout>
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>>.

             --
             You received this message because you are subscribed to

the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%252Bunsubscribe@googlegroups.com>
>
<mailto:elasticsearch+____unsubscribe@googlegroups.com
mailto:elasticsearch%2B__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com
>>.
To view this discussion on the web visit
https://groups.google.com/d/____msgid/elasticsearch/
CANXJSR9-xpGoJR4gvPW%3DKKP%3DVr_____5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-
__xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%__40mail.gmail.com>

    <https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-

__xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%__40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-
xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%40mail.gmail.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/

CANXJSR9-xpGoJR4gvPW%3DKKP%3DVr_____5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com?utm_medium=____email&utm_source=footer
<https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-
xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-

xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com?utm_medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-
xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com?utm_medium=email&utm_source=footer>>>.

             For more options, visit https://groups.google.com/d/__

__optout https://groups.google.com/d/__optout
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.

         --
         Costin

         --
         You received this message because you are subscribed to the

Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%252Bunsubscribe@googlegroups.com>
>.
To view this discussion on the web visit
https://groups.google.com/d/____msgid/elasticsearch/
537C738E.____9030001%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/537C738E.
__9030001%40gmail.com>

         <https://groups.google.com/d/__msgid/elasticsearch/537C738E.

__9030001%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/537C738E.
9030001%40gmail.com>>.

         For more options, visit https://groups.google.com/d/__

__optout https://groups.google.com/d/__optout
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.

    --
    You received this message because you are subscribed to the

Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/
CANXJSR8Gy4qhM8Lzm5BHCr2-__xptLKBSdm00nCrz-JtnW59jWcw%__40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/
CANXJSR8Gy4qhM8Lzm5BHCr2-xptLKBSdm00nCrz-JtnW59jWcw%40mail.gmail.com>
<https://groups.google.com/d/msgid/elasticsearch/
CANXJSR8Gy4qhM8Lzm5BHCr2-xptLKBSdm00nCrz-JtnW59jWcw%
40mail.gmail.com?utm_medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/
CANXJSR8Gy4qhM8Lzm5BHCr2-xptLKBSdm00nCrz-JtnW59jWcw%
40mail.gmail.com?utm_medium=email&utm_source=footer>>.

    For more options, visit https://groups.google.com/d/__optout <

https://groups.google.com/d/optout>.

--
Costin

--
You received this message because you are subscribed to the Google

Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%
2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/_msgid/elasticsearch/537C8751.
_1030908%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/537C8751.
1030908%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <

https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANXJSR_
pj6i55jwA3wYiGgh1d_q86DpcNRKLZzhk-V97vWFLaQ%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CANXJSR_
pj6i55jwA3wYiGgh1d_q86DpcNRKLZzhk-V97vWFLaQ%40mail.gmail.com?utm_medium=
email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/537E0B0B.3070307%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANXJSR9VEYNegZ%3D7AHdz%2Bhm6wBSPe7D5yZPQRd6XwB%3DqWZwqxg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(hanine) #10

by the way I wanna put data here "es.resource' = 'reseaux_sociaux/tweets'"
and I did not create it,shoud I create it first or what ?!!

2014-05-27 21:53 GMT+01:00 hanine haninne haninne.5@gmail.com:

Hello,
I m sorry,but it did not work yet. I thinking I m missing something,Here
is what I did from the beginning ( from getting tweets with flume )

1-script flume (Tweets.conf)
2-store data in HDFS as indicated in the script
3-then I create a table with hive and get data there (in order to have the
possibility to communicate with elasticsearch after) the scripte hive is in
hive_tweets
now I wanna put that data in elasticsearch(head) in order to show it in an
ergonomic interface (like image1 )
so that I tried this script (hiveES)
when I execute it, it gives me :

> STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
> TBLPROPERTIES('es.resource' =

'reseaux_sociaux/tweets','es.input.json' = 'yes');
OK
Time taken: 0.133 seconds

hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201405272117_0002, Tracking URL =
http://master:50030/jobdetails.jsp?jobid=job_201405272117_0002
Kill Command = /home/hduser/hadoop/libexec/../bin/hadoop job -kill
job_201405272117_0002

Hadoop job information for Stage-0: number of mappers: 1; number of
reducers: 0
2014-05-27 21:51:14,893 Stage-0 map = 0%, reduce = 0%
2014-05-27 21:51:35,019 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201405272117_0002 with errors

Error during job, obtaining debugging information...
Job Tracking URL:
http://master:50030/jobdetails.jsp?jobid=job_201405272117_0002
Examining task ID: task_201405272117_0002_m_000002 (and more) from job
job_201405272117_0002

Task with the most failures(4):

Task ID:
task_201405272117_0002_m_000000

URL:

http://master:50030/taskdetails.jsp?jobid=job_201405272117_0002&tipid=task_201405272117_0002_m_000000


Diagnostic Messages for this Task:
java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {"id":467649114269028352,"created_at":"Sat May 17 12:53:32
+0000 2014","source":"<a href="http://www.apple.com"
rel="nofollow">iOS","favorited":false,"retweet_count":0,"retweeted_status":null,"entities":{"urls":[{"expanded_url":"
https://www.google.co.uk/search?q=viera+lifting+fa+cup+for+arsenal&client=safari&hl=en-gb&source=lnms&tbm=isch&sa=X&ei=alp3U4LrK82UOufRgaAK&ved=0CAYQ_AUoAQ&biw=320&bih=460#facrc=&imgrc=-QK5IhwQT8AS8M%253A%3B2yfUWG7-cngS-M%3Bhttp%253A%252F%252Ffarm4.static.flickr.com%252F3777%252F10838930556_618b87d7cc_m.jpg%3Bhttp%253A%252F%252Fwhotalking.com%252Fflickr%252F2-0%252BChelsea%3B1024%3B669"}],"user_mentions":[{"screen_name":"jameswilliams46","name":"James
Williams"}],"hashtags":[]},"text":"@jameswilliams46
https://t.co/CAmxdagAym","user":{"screen_name":"SSimpson7379","name":"Stephen
Simpson","friends_count":258,"followers_count":266,"statuses_count":2609,"verified":false,"utc_offset":3600,"time_zone":"London"},"in_reply_to_screen_name":"jameswilliams46"}
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:162)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row {"id":467649114269028352,"created_at":"Sat May
17 12:53:32 +0000 2014","source":"<a href="http://www.apple.com"
rel="nofollow">iOS","favorited":false,"retweet_count":0,"retweeted_status":null,"entities":{"urls":[{"expanded_url":"
https://www.google.co.uk/search?q=viera+lifting+fa+cup+for+arsenal&client=safari&hl=en-gb&source=lnms&tbm=isch&sa=X&ei=alp3U4LrK82UOufRgaAK&ved=0CAYQ_AUoAQ&biw=320&bih=460#facrc=
&imgrc=-QK5IhwQT8AS8M%253A%3B2yfUWG7-cngS-M%3Bhttp%253A%252F%252Ffarm4.static.flickr.com%252F3777%252F10838930556_618b87d7cc_m.jpg%3Bhttp%253A%252F%252Fwhotalking.com%252Fflickr%252F2-0%252BChelsea%3B1024%3B669"}],"user_mentions":[{"screen_name":"jameswilliams46","name":"James
Williams"}],"hashtags":[]},"text":"@jameswilliams46
https://t.co/CAmxdagAym","user":{"screen_name":"SSimpson7379","name":"Stephen
Simpson","friends_count":258,"followers_count":266,"statuses_count":2609,"verified":false,"utc_offset":3600,"time_zone":"London"},"in_reply_to_screen_name":"jameswilliams46"}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:671)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144)
... 8 more
Caused by: java.lang.IllegalArgumentException: When using JSON input, only
one field is expected
at org.elasticsearch.hadoop.util.Assert.isTrue(Assert.java:48)
at
org.elasticsearch.hadoop.hive.HiveBytesConverter.convert(HiveBytesConverter.java:48)
at
org.elasticsearch.hadoop.serialization.command.JsonTemplatedCommand.preProcess(JsonTemplatedCommand.java:56)
at
org.elasticsearch.hadoop.serialization.command.TemplatedCommand.write(TemplatedCommand.java:73)
at org.elasticsearch.hadoop.hive.EsSerDe.serialize(EsSerDe.java:135)

at

org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:617)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:652)
... 9 more

FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

Any help would be so appreciable
Thank you so much

2014-05-22 15:34 GMT+01:00 Costin Leau costin.leau@gmail.com:

Hi,

It looks like you have two tables - one that uses the JSONSerDe from
cloudera and another one using es-hadoop.
You configured your es-hadoop table to consider the input as json however
it does not receive the proper format (as the exception indicates).
See this [1] section of the documentation for more information.

Let me restate again that you do not the data to Elasticsearch and
es-hadoop to be in JSON format - unless it actually is in JSON
(as in JSON files are stored in HDFS) simply send whatever data you have
in Hive to es-hadoop and it will do the JSON translation automatically
and typically much more efficient.
Again the docs [2] discuss this in detail. Frankly, I'm not sure why you
want to use the JSONSerDe.

To conclude: if the data is in raw format in HDFS, just feed to es-hadoop
with the input configured as JSON, if it's in a different format,
load it in Hive as you would typically do and then insert it into the
es-hadoop table.

Hope this helps,

[1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/
current/hive.html#writing-json-hive
[2] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/
current/hive.html#hive

On 5/22/14 12:13 PM, hanine haninne wrote:

Hello again,
I m trying to get some tweets from the HDFS and put it in
elasticsearch(head) in order to make search on it, I did what
u said and I m so thankful, But it start reading and does not continue.
I don't know why ?!!
here is what I did :

hive -hiveconf hive.aux.jars.path=/ES_Hadoop/
dist/elasticsearch-hadoop-2.0.0.RC1.jar

Logging initialized using configuration in
jar:file:/home/hduser/hadoop/hive-0.11.0/lib/hive-common-0.
11.0.jar!/hive-log4j.properties
Hive history file=/tmp/hduser/hive_job_log_hduser_8166@mouna-pc_
201405220958_2036150447.txt

hive> ADD JAR /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-
SNAPSHOT.jar;
Added /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-SNAPSHOT.jar
to class path
Added resource: /home/hduser/hadoop/hive-0.11.0/lib/hive-serdes-1.0-
SNAPSHOT.jar
hive> INSERT OVERWRITE TABLE tweetsES SELECT * FROM tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201405220937_0005, Tracking URL =
http://master:50030/jobdetails.jsp?jobid=job_201405220937_0005
Kill Command = /home/hduser/hadoop/libexec/../bin/hadoop job -kill
job_201405220937_0005
Hadoop job information for Stage-0: number of mappers: 1; number of
reducers: 0
2014-05-22 10:01:39,184 Stage-0 map = 0%, reduce = 0%
2014-05-22 10:01:59,320 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201405220937_0005 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://master:50030/jobdetails.jsp?jobid=job_
201405220937_0005
Examining task ID: task_201405220937_0005_m_000002 (and more) from job
job_201405220937_0005

Task with the most failures(4):

Task ID:
task_201405220937_0005_m_000000

URL:
http://master:50030/taskdetails.jsp?jobid=job_
201405220937_0005&tipid=task_201405220937_0005_m_000000

Diagnostic Messages for this Task:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row
{"id":467649114269028352,"created_at":"Sat May 17 12:53:32 +0000
2014","source":"<a href="http://www.apple.com"
rel="nofollow">iOS","favorited":false,"retweet_
count":0,"retweeted_status":null,"entities":{"urls":[{"expanded_url":"
https://www.google.co.uk/search?q=viera+lifting+fa+cup+for+arsenal&
client=safari&hl=en-gb&source=lnms&tbm=isch&sa=X&ei=
alp3U4LrK82UOufRgaAK&ved=0CAYQ_AUoAQ&biw=320&bih=460#
facrc=&imgrc=-QK5IhwQT8AS8M%253A%3B2yfUWG7-cngS-M%3Bhttp%
253A%252F%252Ffarm4.static.flickr.com%252F3777%
252F10838930556_618b87d7cc_m.jpg%3Bhttp%253A%252F%
252Fwhotalking.com%252Fflickr%252F2-0%252BChelsea%3B1024%3B669
"}],"user_mentions":[{"screen_name":"jameswilliams46","name":"James
Williams"}],"hashtags":[]},"text":"@jameswilliams46
https://t.co/CAmxdagAym","user":{"screen_name":"
SSimpson7379","name":"Stephen
Simpson","friends_count":258,"followers_count":266,"
statuses_count":2609,"verified":false,"utc_offset":
3600,"time_zone":"London"},"in_reply_to_screen_name":"jameswilliams46"}
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(
ExecMapper.java:162)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(
UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row
{"id":467649114269028352,"created_at":"Sat May 17 12:53:32 +0000
2014","source":"<a href="http://www.apple.com"
rel="nofollow">iOS","favorited":false,"retweet

count":0,"retweeted_status":null,"entities":{"urls":[{"expanded_url":"
https://www.google.co.uk/search?q=viera+lifting+fa+cup+for+arsenal&
client=safari&hl=en-gb&source=lnms&tbm=isch&sa=X&ei=
alp3U4LrK82UOufRgaAK&ved=0CAYQ_AUoAQ&biw=320&bih=460#
facrc=_&imgrc=-QK5IhwQT8AS8M%253A%3B2yfUWG7-cngS-M%3Bhttp%
253A%252F%252Ffarm4.static.flickr.com%252F3777%
252F10838930556_618b87d7cc_m.jpg%3Bhttp%253A%252F%
252Fwhotalking.com%252Fflickr%252F2-0%252BChelsea%3B1024%3B669
"}],"user_mentions":[{"screen_name":"jameswilliams46","name":"James
Williams"}],"hashtags":[]},"text":"@jameswilliams46
https://t.co/CAmxdagAym","user":{"screen_name":"
SSimpson7379","name":"Stephen
Simpson","friends_count":258,"followers_count":266,"
statuses_count":2609,"verified":false,"utc_offset":
3600,"time_zone":"London"},"in_reply_to_screen_name":"jameswilliams46"}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(
MapOperator.java:671)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(
ExecMapper.java:144)
... 8 more
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException:
When using JSON input, only one field is expected
at org.elasticsearch.hadoop.util.Assert.isTrue(Assert.java:50)
at org.elasticsearch.hadoop.hive.HiveBytesConverter.convert(
HiveBytesConverter.java:48)
at org.elasticsearch.hadoop.serialization.bulk.
JsonTemplatedBulk.preProcess(JsonTemplatedBulk.java:58)
at org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.write(
TemplatedBulk.java:75)
at org.elasticsearch.hadoop.hive.EsSerDe.serialize(EsSerDe.
java:128)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.
processOp(FileSinkOperator.java:617)
at org.apache.hadoop.hive.ql.exec.Operator.process(
Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(
Operator.java:832)
at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(
SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(
Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(
Operator.java:832)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.
processOp(TableScanOperator.java:90)
at org.apache.hadoop.hive.ql.exec.Operator.process(
Operator.java:502)
at org.apache.hadoop.hive.ql.exec.Operator.forward(
Operator.java:832)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(
MapOperator.java:652)
... 9 more

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.
exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

when I did not add that jar "ADD JAR /home/hduser/hadoop/hive-0.11.
0/lib/hive-serdes-1.0-SNAPSHOT.jar;" it gave me that
error "Caused by: java.lang.ClassNotFoundException:
com.cloudera.hive.serde.JSONSerDe
"

Thank you so very much for your helping

2014-05-21 12:00 GMT+01:00 Costin Leau <costin.leau@gmail.com <mailto:
costin.leau@gmail.com>>:

If you plan to use es-hadoop, there's no need to do any JSON

conversion (es-hadoop does that automatically) - see
the documentation at [1]
for more information.
If not than I recommend reaching out to the providers of the package
for more information on its configuration.

[1] elasticsearch.org/hadoop <http://elasticsearch.org/hadoop>



On 5/21/14 1:17 PM, hanine haninne wrote:

    yes, you are right.Thanks so much

    well I tried hive with HDFS and it works very well,by the way I

m using hive-0.11.0 , hadoop-1.2.1 and
elasticsearch-1.0.1
Here is the script of the tables
CREATE EXTERNAL TABLE tweets (
id BIGINT,
created_at STRING,
source STRING,
favorited BOOLEAN,
retweet_count INT,
retweeted_status STRUCT<
text:STRING,
user:STRUCT<screen_name:__STRING,name:STRING>>,
entities STRUCT<
urls:ARRAY<STRUCT<expanded___url:STRING>>,
user_mentions:ARRAY<STRUCT<screen_name:STRING,name:
STRING>>,
hashtags:ARRAY<STRUCTtext:__STRING>>,

         text STRING,
         user STRUCT<
            screen_name:STRING,
            name:STRING,
            friends_count:INT,
            followers_count:INT,
            statuses_count:INT,
            verified:BOOLEAN,
            utc_offset:INT,
            time_zone:STRING>,
         in_reply_to_screen_name STRING
    )
    ROW FORMAT SERDE 'com.cloudera.hive.serde.__JSONSerDe'
    LOCATION '/flume/Tweets_A';


    do I have to add "com.cloudera.hive.serde.__JSONSerDe'" an

other time to put data in elasticsearch ?? if yes,How !!

    Thank you so very much


    2014-05-21 10:36 GMT+01:00 Costin Leau <costin.leau@gmail.com<mailto:

costin.leau@gmail.com>
<mailto:costin.leau@gmail.com mailto:costin.leau@gmail.com

__>:

         The error indicates that your script refers to a JSonSerDe

class (presumably from Cloudera) which cannot be
found in
the classpath:

         "> Caused by: java.lang.____ClassNotFoundException:

com.cloudera.hive.serde.____JSONSerDe"

         In fact if you read through the stacktrace, you'll notice

that this comes up while the job configuration
is being read and this class name encountered but the class
definition itself cannot be resolved.

         I'm not sure what causes this class to be added to the

classpath of your script - it could be your script
or the
client configuration.
Note the class is not used or added by ES-hadoop itself.

         I'm not sure what distro you are using (es-hadoop works

across Apache Hadoop, Cloudera, HDP, MapR, Pivotal,
etc...)
but you could try running some
basic Hive scripts (without es-hadoop) first to make sure
the environment is sane and that you can properly
read and
write from HDFS and then add
es-hadoop into the mix.

         Hope this helps,


         On 5/21/14 12:15 PM, hanine haninne wrote:

             Thank you so much for ur answer
             here is what it gives me now

                 INSERT OVERWRITE TABLE tweetsES SELECT * FROM

tweets;

             Total MapReduce jobs = 1
             Launching Job 1 out of 1
             Number of reduce tasks is set to 0 since there's no

reduce operator
Starting Job = job_201405210900_0002, Tracking URL =
http://master:50030/jobdetails.jsp?jobid=job_
201405210900_0002

    <http://master:50030/__jobdetails.jsp?jobid=job___

201405210900_0002>
<http://master:50030/jobdetails.jsp?jobid=job_
201405210900_0002
<http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002

             Kill Command = /home/hduser/hadoop/libexec/..____/bin/hadoop

job -kill job_201405210900_0002

             Hadoop job information for Stage-0: number of mappers:

1; number of reducers: 0
2014-05-21 10:12:05,285 Stage-0 map = 0%, reduce = 0%
2014-05-21 10:12:23,408 Stage-0 map = 100%, reduce =
100%
Ended Job = job_201405210900_0002 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://master:50030/____
jobdetails.jsp?jobid=job_____201405210900_0002
<http://master:50030/jobdetails.jsp?jobid=job_
201405210900_0002>

             <http://master:50030/__jobdetails.jsp?jobid=job___

201405210900_0002
<http://master:50030/jobdetails.jsp?jobid=job_201405210900_0002

             Examining task ID: task_201405210900_0002_m_____000002

(and more) from job job_201405210900_0002

             Task with the most failures(4):
             -----
             Task ID:
                 task_201405210900_0002_m_____000000

             URL:
    http://master:50030/____taskdetails.jsp?jobid=job_____

201405210900_0002&tipid=task_____201405210900_0002_m_000000 <
http://master:50030/taskdetails.jsp?jobid=job_
201405210900_0002&tipid=task___201405210900_0002_m_000000>

    <http://master:50030/__taskdetails.jsp?jobid=job___

201405210900_0002&tipid=task___201405210900_0002_m_000000
<http://master:50030/taskdetails.jsp?jobid=job_
201405210900_0002&tipid=task_201405210900_0002_m_000000>>
-----
Diagnostic Messages for this Task:
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.____
ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.

ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.

ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.

MapTask.runOldMapper(MapTask.java:426)
at org.apache.hadoop.mapred.

MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.____
Child$4.run(Child.java:255)

                   at java.security.____

AccessController.doPrivileged(_Native Method)
at javax.security.auth.Subject.

doAs(Subject.java:415)
at org.apache.hadoop.security.

_UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.

Child.main(Child.java:249)
Caused by: java.lang.reflect.

InvocationTargetException

                   at sun.reflect.____NativeMethodAccessorImpl.____invoke0(Native

Method)
at sun.reflect.NativeMethodAccessorImpl.
invoke(____NativeMethodAccessorImpl.java:____57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.

invoke(Method.java:606)
at org.apache.hadoop.util.

ReflectionUtils.setJobConf(____ReflectionUtils.java:88)

                   ... 9 more
             Caused by: java.lang.RuntimeException: Error in

configuring object
at org.apache.hadoop.util.____
ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.

ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.

ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.

MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.

InvocationTargetException

                   at sun.reflect.____NativeMethodAccessorImpl.____invoke0(Native

Method)
at sun.reflect.NativeMethodAccessorImpl.
invoke(____NativeMethodAccessorImpl.java:____57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.

invoke(Method.java:606)
at org.apache.hadoop.util.

ReflectionUtils.setJobConf(____ReflectionUtils.java:88)

                   ... 17 more
             Caused by: java.lang.RuntimeException: Map operator

initialization failed
at org.apache.hadoop.hive.ql.____
exec.ExecMapper.configure(____ExecMapper.java:121)
... 22 more
Caused by: org.apache.hadoop.hive.ql.____metadata.HiveException:
java.lang.____ClassNotFoundException:
com.cloudera.hive.serde.JSONSerDe
at org.apache.hadoop.hive.ql.

exec.MapOperator.____getConvertedOI(MapOperator.java:463)
at org.apache.hadoop.hive.ql.

exec.MapOperator.setChildren(MapOperator.java:479)
at org.apache.hadoop.hive.ql.

exec.ExecMapper.configure(____ExecMapper.java:90)
... 22 more
Caused by: java.lang.____ClassNotFoundException:
com.cloudera.hive.serde.____JSONSerDe
at java.net.URLClassLoader$1.run(
____URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(
URLClassLoader.java:355)
at java.security.

AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.

findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.

loadClass(ClassLoader.java:424)
at sun.misc.Launcher$

AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.

loadClass(ClassLoader.java:____357)
at java.lang.Class.forName0(____Native Method)
at java.lang.Class.forName(Class.java:270)
at org.apache.hadoop.conf.

Configuration.getClassByName(Configuration.java:810)
at org.apache.hadoop.hive.ql.

exec.MapOperator.____getConvertedOI(MapOperator.____java:422)
... 24 more

             FAILED: Execution Error, return code 2 from

org.apache.hadoop.hive.ql.____exec.MapRedTask

             MapReduce Jobs Launched:
             Job 0: Map: 1   HDFS Read: 0 HDFS Write: 0 FAIL
             Total MapReduce CPU Time Spent: 0 msec



             2014-05-20 14:11 GMT+01:00 Costin Leau <

costin.leau@gmail.com mailto:costin.leau@gmail.com
<mailto:costin.leau@gmail.com mailto:costin.leau@gmail.com>
<mailto:costin.leau@gmail.com <mailto:
costin.leau@gmail.com> <mailto:costin.leau@gmail.com
mailto:costin.leau@gmail.com>>>:

                  1. You don't need to add the hive jars

(hive-serdes) to your script - these are already part of
the runtime
2. use es-hadoop-2.0.RC1, 1.3 M2 is fairly old
3. Make sure that the jar is actually available
and that is potentially accessible by Hive. Also
do note that
when running Hive against a cluster, it's best to
have the jars available on all nodes hence why
most folks
copy
them to HDFS and refer to that location.
While /home/hduser might be locally available, it
might not exist on the Hive server - as
indicated by the
error.

                  Hope this helps,


                  On 5/20/14 3:58 PM, hanine haninne wrote:


                      Hello everybody

                      I m trying to take data from Hive and put it

in elasticsearch
here is the script and the error

                      hive> ADD JAR /home/hduser/hadoop/hive-0.11.

______0/lib/hive-serdes-1.0-______SNAPSHOT.jar;
Added /home/hduser/hadoop/hive-0.11.
______0/lib/hive-serdes-1.0-______SNAPSHOT.jar to class path
Added resource: /home/hduser/hadoop/hive-0.11.
______0/lib/hive-serdes-1.0-______SNAPSHOT.jar

                      hive> CREATE EXTERNAL TABLE tweetsES (
                            >    id BIGINT,
                            >    created_at STRING,
                            >    source STRING,
                            >    favorited BOOLEAN,
                            >    retweet_count INT,
                            >    retweeted_status STRUCT<
                            >       text:STRING,
                            >       user:STRUCT<screen_name:______

STRING,name:STRING>>,
> entities STRUCT<
> urls:ARRAY<STRUCT<expanded____
url:STRING>>,
> user_mentions:ARRAY<STRUCT<

___screen_name:STRING,name:___STRING>>,
> hashtags:ARRAY<STRUCT<text:

___STRING>>>,

                            >    text STRING,
                            >    user STRUCT<
                            >       screen_name:STRING,
                            >       name:STRING,
                            >       friends_count:INT,
                            >       followers_count:INT,
                            >       statuses_count:INT,
                            >       verified:BOOLEAN,
                            >       utc_offset:INT,
                            >       time_zone:STRING>,
                            >    in_reply_to_screen_name STRING
                            > )
                            > STORED BY

'org.elasticsearch.hadoop.______hive.EsStorageHandler'
> TBLPROPERTIES('es.resource' =
'reseaux_sociaux/tweets','es.______input.json' = 'yes');

                      OK
                      Time taken: 4.292 seconds
                      hive> ADD JAR /home/hduser/hadoop/______

elasticsearch-hadoop-1.3.0.M2.jar;
Added /home/hduser/hadoop/

elasticsearch-hadoop-1.3.0.M2.jar to class path
Added resource: /home/hduser/hadoop/

elasticsearch-hadoop-1.3.0.M2.______jar

                      hive> INSERT OVERWRITE TABLE tweetsES SELECT *

FROM tweets;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since
there's no reduce operator
java.io.FileNotFoundException: File does not
exist:
/home/hduser/hadoop/______
elasticsearch-hadoop-1.3.0.M2.______jar
at

    org.apache.hadoop.hdfs.______DistributedFileSystem.______

getFileStatus(______DistributedFileSystem.java:______558)
at
org.apache.hadoop.filecache.DistributedCache.
getFileStatus(______DistributedCache.java:185)
at

    org.apache.hadoop.filecache.________

TrackerDistributedCacheManager______.getFileStatus(______
TrackerDistributedCacheManager______.java:723)
at

    org.apache.hadoop.filecache.________

TrackerDistributedCacheManager______.determineTimestamps(______
TrackerDistributedCacheManager______.java:778)
at

    org.apache.hadoop.filecache.________

TrackerDistributedCacheManager______.______
determineTimestampsAndCacheVis______ibilities(______
TrackerDistributedCacheManager______.java:755)
at
org.apache.hadoop.mapred.JobClient.
copyAndConfigureFiles(______JobClient.java:843)
at
org.apache.hadoop.mapred.JobClient.
copyAndConfigureFiles(_JobClient.java:734)
at org.apache.hadoop.mapred.

_JobClient.access$400(_JobClient.java:179)
at org.apache.hadoop.mapred.

_JobClient$2.run(JobClient._java:951)
at org.apache.hadoop.mapred.

_JobClient$2.run(JobClient.java:936)
at java.security.

AccessController.doPrivileged(____Native Method)
at javax.security.auth.Subject.

____doAs(Subject.java:415)
at
org.apache.hadoop.security._____UserGroupInformation.doAs(
_____UserGroupInformation.java:_1190)
at org.apache.hadoop.mapred.

_JobClient.submitJobInternal(_JobClient.java:936)
at org.apache.hadoop.mapred.

_JobClient.submitJob(JobClient.__java:910)
at org.apache.hadoop.hive.ql.

__exec.ExecDriver.execute(__ExecDriver.java:447)
at org.apache.hadoop.hive.ql.

__exec.MapRedTask.execute(__MapRedTask.java:138)
at org.apache.hadoop.hive.ql.

__exec.Task.executeTask(Task.__java:144)
at org.apache.hadoop.hive.ql.

__exec.TaskRunner.runSequential(__TaskRunner.java:57)
at org.apache.hadoop.hive.ql.

__Driver.launchTask(Driver.java:__1355)
at org.apache.hadoop.hive.ql.

__Driver.execute(Driver.java:_1139)
at org.apache.hadoop.hive.ql.

Driver.run(Driver.java:945)
at org.apache.hadoop.hive.cli.

___CliDriver.processLocalCmd(___CliDriver.java:259)
at org.apache.hadoop.hive.cli.

___CliDriver.processCmd(___CliDriver.java:216)
at org.apache.hadoop.hive.cli.

___CliDriver.processLine(___CliDriver.java:413)
at org.apache.hadoop.hive.cli.

___CliDriver.run(CliDriver.java:___756)
at org.apache.hadoop.hive.cli.

___CliDriver.main(CliDriver.java:614)
at sun.reflect.

NativeMethodAccessorImpl.______invoke0(Native Method)
at
sun.reflect.______NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:______57)
at

    sun.reflect.______DelegatingMethodAccessorImpl._

_____invoke(______DelegatingMethodAccessorImpl._java:43)
at java.lang.reflect.Method.

_invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.
______main(RunJar.java:160)

                      Job Submission failed with exception 'java.io<

http://java.io> http://java.io
http://java.io.FileNotFoundException(File
does not exist:
/home/hduser/hadoop/

elasticsearch-hadoop-1.3.0.M2.______jar)'
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.______exec.MapRedTask

                      That path it exist :/home/hduser/hadoop/______

elasticsearch-hadoop-1.3.0.M2.______jar

                      Any help would be so appreciable
                      Thanks

                      --
                      You received this message because you are

subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop
receiving emails from it, send an email to
elasticsearch+unsubscribe@go
oglegroups.com http://go__oglegroups.com
http://googlegroups.com
<mailto:elasticsearch%

2Bunsubscribe@googlegroups.com
mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
<mailto:elasticsearch%252Bunsubscribe@googlegroups.com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>
>
<mailto:elasticsearch+
__
unsubscribe@googlegroups.com
mailto:elasticsearch%2B____unsubscribe@googlegroups.com
<mailto:elasticsearch%2B____
unsubscribe@googlegroups.com
mailto:elasticsearch%2B__unsubscribe@googlegroups.com>
<mailto:elasticsearch%____2Bunsubscribe@googlegroups.com<mailto:
elasticsearch%25__2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch%252Bunsubscribe@googlegroups.
_com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>
_>>.

                      To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__

d1613b48-____e58d-4077-a579-__92b46a97bcff%______40googlegroups.com
<https://groups.google.com/d/____msgid/elasticsearch/
d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com>

    <https://groups.google.com/d/____msgid/elasticsearch/

d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/d1613b48-
__e58d-4077-a579-92b46a97bcff%__40googlegroups.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/

d1613b48-____e58d-4077-a579-92b46a97bcff%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/d1613b48-
__e58d-4077-a579-92b46a97bcff%__40googlegroups.com>

    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-

__e58d-4077-a579-92b46a97bcff%__40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/d1613b48-
e58d-4077-a579-92b46a97bcff%40googlegroups.com>>>

    <https://groups.google.com/d/______msgid/elasticsearch/__

d1613b48-e58d-4077-a579-92b46a97bcff%
40googlegroups.com?utm_____medium=__email&utm_source=____footer
<https://groups.google.com/d/____msgid/elasticsearch/
d1613b48-____e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/

d1613b48-____e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/d1613b48-
__e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=footer>>

    <https://groups.google.com/d/____msgid/elasticsearch/

d1613b48-____e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/d1613b48-
__e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/d1613b48-

__e58d-4077-a579-92b46a97bcff%_40googlegroups.com?utm
medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/d1613b48-
e58d-4077-a579-92b46a97bcff%40googlegroups.com?utm_medium=
email&utm_source=footer>>>>.

                      For more options, visit

https://groups.google.com/d/______optout
https://groups.google.com/d/____optout <
https://groups.google.com/d/____optout

    <https://groups.google.com/d/__optout>>

             <https://groups.google.com/d/____optout <

https://groups.google.com/d/__optout>
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>>.

                  --
                  Costin


                  --
                  You received this message because you are

subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving
emails from it, send an email to
elasticsearch+unsubscribe@go____oglegroups.com <
http://go__oglegroups.com> http://googlegroups.com
<mailto:elasticsearch%
__
2Bunsubscribe@googlegroups.com
mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
<mailto:elasticsearch%__252Bunsubscribe@googlegroups.com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>
>.

                  To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__

537B549F.____8020501%40gmail.__com
<https://groups.google.com/d/____msgid/elasticsearch/
537B549F.____8020501%40gmail.com>

             <https://groups.google.com/d/____msgid/elasticsearch/

537B549F.____8020501%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/537B549F.
__8020501%40gmail.com>>

                  <https://groups.google.com/d/_

___msgid/elasticsearch/537B549F.____8020501%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/537B549F.
_8020501%40gmail.com>
<https://groups.google.com/d/

_msgid/elasticsearch/537B549F.__8020501%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/537B549F.
8020501%40gmail.com>>>.

                  For more options, visit

https://groups.google.com/d/______optout
https://groups.google.com/d/____optout <
https://groups.google.com/d/____optout
https://groups.google.com/d/__optout>

             <https://groups.google.com/d/____optout <

https://groups.google.com/d/__optout>
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>>.

             --
             You received this message because you are subscribed to

the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving
emails from it, send an email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%252Bunsubscribe@googlegroups.com>
>
<mailto:elasticsearch+____unsubscribe@googlegroups.com
mailto:elasticsearch%2B__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com
>>.
To view this discussion on the web visit
https://groups.google.com/d/____msgid/elasticsearch/
CANXJSR9-xpGoJR4gvPW%3DKKP%3DVr_____5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-
__xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%__40mail.gmail.com>

    <https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-

__xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%__40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-
xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%40mail.gmail.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/

CANXJSR9-xpGoJR4gvPW%3DKKP%3DVr_____5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com?utm_medium=____email&utm_source=footer
<https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-
xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/CANXJSR9-

xpGoJR4gvPW%3DKKP%3DVr___5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com?utm_medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/CANXJSR9-
xpGoJR4gvPW%3DKKP%3DVr_5MxyDoUJKtbpqM6CsYs1G4Q%
40mail.gmail.com?utm_medium=email&utm_source=footer>>>.

             For more options, visit https://groups.google.com/d/__

__optout https://groups.google.com/d/__optout
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.

         --
         Costin

         --
         You received this message because you are subscribed to the

Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%252Bunsubscribe@googlegroups.com>
>.
To view this discussion on the web visit
https://groups.google.com/d/____msgid/elasticsearch/
537C738E.____9030001%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/537C738E.
__9030001%40gmail.com>

         <https://groups.google.com/d/_

_msgid/elasticsearch/537C738E.__9030001%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/537C738E.
9030001%40gmail.com>>.

         For more options, visit https://groups.google.com/d/__

__optout https://groups.google.com/d/__optout
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.

    --
    You received this message because you are subscribed to the

Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/
CANXJSR8Gy4qhM8Lzm5BHCr2-__xptLKBSdm00nCrz-JtnW59jWcw%__40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/
CANXJSR8Gy4qhM8Lzm5BHCr2-xptLKBSdm00nCrz-JtnW59jWcw%40mail.gmail.com>
<https://groups.google.com/d/msgid/elasticsearch/
CANXJSR8Gy4qhM8Lzm5BHCr2-xptLKBSdm00nCrz-JtnW59jWcw%
40mail.gmail.com?utm_medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/
CANXJSR8Gy4qhM8Lzm5BHCr2-xptLKBSdm00nCrz-JtnW59jWcw%
40mail.gmail.com?utm_medium=email&utm_source=footer>>.

    For more options, visit https://groups.google.com/d/__optout <

https://groups.google.com/d/optout>.

--
Costin

--
You received this message because you are subscribed to the Google

Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%
2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/_msgid/elasticsearch/537C8751.
_1030908%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/537C8751.
1030908%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <

https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANXJSR_
pj6i55jwA3wYiGgh1d_q86DpcNRKLZzhk-V97vWFLaQ%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CANXJSR_
pj6i55jwA3wYiGgh1d_q86DpcNRKLZzhk-V97vWFLaQ%40mail.gmail.com?utm_medium=
email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/537E0B0B.3070307%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANXJSR-yASyzyEDUokfstUm%3Dco64JepijaMm8ti05w7tw7qk2Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #11