# Composite key for creating elastic search Index

**URL:** <https://discuss.elastic.co/t/composite-key-for-creating-elastic-search-index/100221>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [September 12, 2017, 2:11pm UTC](https://discuss.elastic.co/t/composite-key-for-creating-elastic-search-index/100221 "2017-09-12T14:11:38Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sachin111us](https://avatars.discourse-cdn.com/v4/letter/s/b19c9b/32.png) [@Sachin111us](https://discuss.elastic.co/u/Sachin111us)\
**Post date:** [September 12, 2017, 2:11pm UTC](https://discuss.elastic.co/t/composite-key-for-creating-elastic-search-index/100221/1 "2017-09-12T14:11:38Z")

</div>

Hi,  
I am working on HDFS to Elastic search integration via SparkSQL. I could able to read the csv data from HDFS and create the elastic search index. To create Elastic search Index ID I am using one of the unique column from the csv data. Now my requirement is Elastic search Index ID should be combination of 2 CSV columns. Does anybody aware how would I achieve this? I am using elasticsearch-spark library to create index. Below is the sample code.

SparkSession sparkSession = SparkSession.builder().config(config).getOrCreate();  
SQLContext ctx = sparkSession.sqlContext();  
HashMap\<String, String\> options = new HashMap\<String, String\>();  
options.put("header", "true");  
options.put("path", "hdfs://localhost:9000/test");  
Dataset df = ctx.read().format("com.databricks.spark.csv").options(options).load();  
JavaEsSparkSQL.saveToEs(df, "spark/test", ImmutableMap.of("[es.mapping.id](http://es.mapping.id)", "Id"));

Thanks  
Sach

---

<div class="post-metadata">

**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\
**Post date:** [October 4, 2017, 3:23am UTC](https://discuss.elastic.co/t/composite-key-for-creating-elastic-search-index/100221/2 "2017-10-04T03:23:56Z")

</div>

In this case, the best option would be to use a Spark transformation to create the composite ID as a field on each record, and tell the connector to use that new composite field as the document's ID.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 1, 2017, 3:24am UTC](https://discuss.elastic.co/t/composite-key-for-creating-elastic-search-index/100221/3 "2017-11-01T03:24:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.