# ES-Hadoop 2.0.2 jars and INSERT OVERWRITE **URL:** \ **Category:** Elasticsearch\ **Tags:** es-hadoop\ **Created:** [June 22, 2015, 1:08pm UTC](https://discuss.elastic.co/t/es-hadoop-2-0-2-jars-and-insert-overwrite/24091 "2015-06-22T13:08:20Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![aken](https://avatars.discourse-cdn.com/v4/letter/a/b38774/32.png) [@aken](https://discuss.elastic.co/u/aken)\ **Post date:** [June 22, 2015, 1:08pm UTC](https://discuss.elastic.co/t/es-hadoop-2-0-2-jars-and-insert-overwrite/24091/1 "2015-06-22T13:08:20Z")
Are there known issues with the 2.0.2 jar and INSERT OVERWRITE? I have defined my external table and can insert into my ES index and everything works like a charm. However, I notice that even when I specify OVERWRITE that the index is always appended. I can of course delete the index before I start, but I would prefer to be able to do from within the Hive context. Thanks in advance, Andrew P.S. I'm using ES 1.5.0, Hive 0.10 (and CDH 4.7). ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [June 22, 2015, 1:24pm UTC](https://discuss.elastic.co/t/es-hadoop-2-0-2-jars-and-insert-overwrite/24091/2 "2015-06-22T13:24:11Z")
Hive doesn't expose the overwrite to external tables. So a different provider is not aware whether an insert is `normal` or `not`. Further more, the SQL semantics are somewhat different - in some case `INSERT OVERWRITE` removes the entire data set but more often than not, only overwrites the entries specified. Thus the object identity need to be defined which is handled by the connector directly, regardless of the `OVERWRITE` or not. In other words, if the write operation is `update` vs `index` vs `create` and the document id is specified, the behaviour can be tweak per entry/doc-level which is typically what one wants. If not, one can simply drop the index before insertion. Hope this helps, ---
**Author:** ![aken](https://avatars.discourse-cdn.com/v4/letter/a/b38774/32.png) [@aken](https://discuss.elastic.co/u/aken)\ **Post date:** [June 22, 2015, 1:32pm UTC](https://discuss.elastic.co/t/es-hadoop-2-0-2-jars-and-insert-overwrite/24091/3 "2015-06-22T13:32:04Z")
Hallo Costin, Thanks for your reply. The standard Hive behaviour with internal tables (certainly with partitions) is that Hive empties the target location/partition and writes the new data afresh. With EXTERNAL TABLES that is a grey area as the data does not strictly "belong" to Hive, so I can understand why a delete does not happen (I would think the most elegant implementation would be to control this behaviour via a property, as is done with e.g.'es.index.auto.create', although DELETEs are there in Hive 0.14). But at least I know that is expected behaviour! Thanks for your help. Andrew ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [June 22, 2015, 1:54pm UTC](https://discuss.elastic.co/t/es-hadoop-2-0-2-jars-and-insert-overwrite/24091/4 "2015-06-22T13:54:32Z")
The problem with properties is that they are table defined. While `OVERWRITE` is query defined - there's no way for a `TABLE` to know whether an `INSERT` is actually `OVERWRITE` or not and deleting the index on each `INSERT` is not a solution. The only way to fix this, especially for destructive operations like `DELETE` is to tell the `STORAGE` about what operation to execute instead of letting it execute it. As a side note, Spark SQL offers such a hook which the connector plugs into and thus understands when an `OVERWRITE` is happening and thus trigger an index delete. From the connector perspective, having such an interface (along with proper pushdown operations) would be great since ultimately will create a better integration and richer experience for using Elasticsearch in Hive. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:28pm UTC](https://discuss.elastic.co/t/es-hadoop-2-0-2-jars-and-insert-overwrite/24091/5 "2017-07-06T13:28:14Z")