# Generating custom \_id when exporting from hadoop/spark to ES

**URL:** <https://discuss.elastic.co/t/generating-custom-id-when-exporting-from-hadoop-spark-to-es/308486>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [June 29, 2022, 2:47pm UTC](https://discuss.elastic.co/t/generating-custom-id-when-exporting-from-hadoop-spark-to-es/308486 "2022-06-29T14:47:45Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Oriol\_Lopez\_Massague](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oriol_lopez_massague/32/105534_2.png) [@Oriol\_Lopez\_Massague](https://discuss.elastic.co/u/Oriol_Lopez_Massague)\
**Post date:** [June 29, 2022, 2:47pm UTC](https://discuss.elastic.co/t/generating-custom-id-when-exporting-from-hadoop-spark-to-es/308486/1 "2022-06-29T14:47:45Z")

</div>

I want to insert data in ES using hadoop/spark but with a custom made \_id field (not the autogerated one). I have seen that there is method using RDDs "saveToEsWithMeta":

> <https://github.com/elastic/elasticsearch-hadoop/blob/969eb3eb2dcff50174ae563e345fde8a4d605316/spark/core/src/main/scala/org/elasticsearch/spark/rdd/EsSpark.scala>

Is there a way to do the same thing using the a dataframe?

---

<div class="post-metadata">

**Author:** ![Keith\_Massey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keith_massey/32/83666_2.png) [@Keith\_Massey](https://discuss.elastic.co/u/Keith_Massey)\
**Post date:** [June 30, 2022, 2:17pm UTC](https://discuss.elastic.co/t/generating-custom-id-when-exporting-from-hadoop-spark-to-es/308486/2 "2022-06-30T14:17:25Z")

</div>

I think that `es.mapping.id` is what you want. Take a look at [Apache Spark support | Elasticsearch for Apache Hadoop [master] | Elastic](https://www.elastic.co/guide/en/elasticsearch/hadoop/master/spark.html#spark-write) and [Configuration | Elasticsearch for Apache Hadoop [8.3] | Elastic](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html#cfg-mapping).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 28, 2022, 2:18pm UTC](https://discuss.elastic.co/t/generating-custom-id-when-exporting-from-hadoop-spark-to-es/308486/3 "2022-07-28T14:18:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.