# Error while inserting data from Hadoop to ES
**URL:** https://discuss.elastic.co/t/error-while-inserting-data-from-hadoop-to-es/219371
**Category:** Elasticsearch
**Tags:** es-hadoop
**Created:** [February 14, 2020, 1:20pm UTC](https://discuss.elastic.co/t/error-while-inserting-data-from-hadoop-to-es/219371 "2020-02-14T13:20:11Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@zizake](https://discuss.elastic.co/u/zizake)
#### Post date: [February 14, 2020, 1:20pm UTC](https://discuss.elastic.co/t/error-while-inserting-data-from-hadoop-to-es/219371/1 "2020-02-14T13:20:12Z")
Hello,
ES: 6.2.3
Hadoop Plugin 6.2.3
I have an external table I want to be populated with some data from a Hive table (Parquet files)
When I execute:
INSERT OVERWRITE TABLE external\_es SELECT `field-name1`,`field-name2`,...,`field-name20` FROM parquet limit 999000000;
I get:
_Status: Failed_
_Vertex failed, vertexName=Reducer 2, vertexId=vertex\_1573460958877\_2186\_1\_01, diagnostics=[Task failed, taskId=task\_1573460958877\_2186\_1\_01\_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{},"value":{"\_col0":"1.0","\_col1":4,"\_col2":1581674662000,"\_col3":"","\_col4":"d4b5cf34f7d0ee4a15c0b231f3e2aaa2","\_col5":"8255bb0f2594fc19a3f752286f8a5a76","\_col6":100,"\_col7":"XXXXXXXXXXXXXXXXXXx","\_col8":"","\_col9":"","\_col10":"10.0.18362.256","\_col11":"","\_col12":"","\_col13":"XXXXXXXXXXXXXXXXXXx","\_col14":null,"\_col15":"clean","\_col16":"12.22.197.186","\_col17":"ore.amz","\_col18":"xxxx","\_col19":"am\_engines","\_col20":"61dc8483-c8bc-402b-a325-a4dc6fb10567","\_col21":"e9ebd55d-2b3e-4291-9ba9-d966e619d4c4","\_col22":null,"\_col23":null,"\_col24":null,"\_col25":true,"\_col26":"x64","\_col27":"XXX","\_col28":{"lat":34.281,"lon":-119.1702},"\_col29":{"region\_name":"California","country\_code":"US","country\_name":"United States","city":"Ventura","continent\_code":"NA","asn":3421,"organization":"Company"}}}_
_at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)_
_at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)_
_at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)_
_at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)_
_at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)_
_at java.security.AccessController.doPrivileged(Native Method)_
_at javax.security.auth.Subject.doAs(Subject.java:422)_
_at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)_
_at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)_
_at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)_
_at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)_
_at java.util.concurrent.FutureTask.run(FutureTask.java:266)_
_at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)_
_at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)_
_at java.lang.Thread.run(Thread.java:745)_
_Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{},"value":{"\_col0":"1.0","\_col1":4,"\_col2":1581674662000,"\_col3":"","\_col4":"d4b5cf34f7d0ee4a15c0b231f3e2aaa2","\_col5":"8255bb0f2594fc19a3f752286f8a5a76","\_col6":100,"\_col7":"","\_col8":"","\_col9":"","\_col10":"10.0.18362.256","\_col11":"","\_col12":"","\_col13":"213123241234412#fdfdf","\_col14":null,"\_col15":"clean","\_col16":"12.22.197.186","\_col17":"ore.amz","\_col18":"xxxx","\_col19":"fact","\_col20":"61dc8483-c8bc-402b-a325-a4dc6fb10567","\_col21":"e9ebd55d-2b3e-4291-9ba9-d966e619d4c4","\_col22":null,"\_col23":null,"\_col24":null,"\_col25":true,"\_col26":"x64","\_col27":"XXXX","\_col28":{"lat":34.281,"lon":-119.1702},"\_col29":{"region\_name":"California","country\_code":"US","country\_name":"United States","city":"Ventura","continent\_code":"NA","asn":3212,"organization":"Company"}}}_
_at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)_
_at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:237)_
_at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:266)_
_at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)_
_... 14 more_
_Caused by: org.elasticsearch.hadoop.EsHadoopException: Could not write all entries for bulk operation [1/951]. Error sample (first [5] error messages):_
_failed to parse [origin\_ip]_
_Bailing out..._
_at org.elasticsearch.hadoop.rest.bulk.BulkProcessor.flush(BulkProcessor.java:475)_
_at org.elasticsearch.hadoop.rest.bulk.BulkProcessor.add(BulkProcessor.java:106)_
_at org.elasticsearch.hadoop.rest.RestRepository.doWriteToIndex(RestRepository.java:187)_
_at org.elasticsearch.hadoop.rest.RestRepository.writeProcessedToIndex(RestRepository.java:183)_
_at org.elasticsearch.hadoop.hive.EsHiveOutputFormat$EsHiveRecordWriter.write(EsHiveOutputFormat.java:63)_
_at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:763)_
_at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:841)_
_at org.apache.hadoop.hive.ql.exec.LimitOperator.process(LimitOperator.java:54)_
_at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:841)_
_at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)_
_at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)_
What I actually don't understand is why I receive this error after inserting with success about 9M rows into ES index (the mapping in ES is IP).
It says _failed to parse [origin\_ip]_. origin\_ip field corresponds to "\_col16" and as it can be seen in JSON it's a valid IP ("\_col16":"12.22.197.186").
Does anyone have a clue ?
Thanks!
---
### Author:  [@zizake](https://discuss.elastic.co/u/zizake)
#### Post date: [February 17, 2020, 9:18am UTC](https://discuss.elastic.co/t/error-while-inserting-data-from-hadoop-to-es/219371/2 "2020-02-17T09:18:49Z")
@costin I look forward to your opinion 🙂 .
Thanks.
---
### Author:  [@james.baiera](https://discuss.elastic.co/u/james.baiera)
#### Post date: [February 27, 2020, 4:01pm UTC](https://discuss.elastic.co/t/error-while-inserting-data-from-hadoop-to-es/219371/3 "2020-02-27T16:01:42Z")
@zizake Please do not ping team members who are not part of the discussion already.
The issue you're seeing seems to be a remote exception from Elasticsearch, specifically that it does not understand the data given to it. I would check your mappings to make sure that the field can be accepted as a string value and that it isn't expecting something else.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 26, 2020, 4:01pm UTC](https://discuss.elastic.co/t/error-while-inserting-data-from-hadoop-to-es/219371/4 "2020-03-26T16:01:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.