# ElasticSearch Spark

**URL:** <https://discuss.elastic.co/t/elasticsearch-spark/61610>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [September 27, 2016, 1:44pm UTC](https://discuss.elastic.co/t/elasticsearch-spark/61610 "2016-09-27T13:44:41Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![nayan\_sharma](https://avatars.discourse-cdn.com/v4/letter/n/d6d6ee/32.png) [@nayan\_sharma](https://discuss.elastic.co/u/nayan_sharma)\
**Post date:** [September 27, 2016, 1:44pm UTC](https://discuss.elastic.co/t/elasticsearch-spark/61610/1 "2016-09-27T13:44:41Z")

</div>

I have a dataframe and writing it to the ES

Before writing to ES,I am converting the EVTExit column to Date, which is in EPOCH.

`**workset = workset.withColumn("EVTExit", to_date(from_unixtime($"EVTExit".divide(1000))))**`

workset.select("EVTExit").show(10)  
+----------+  
| EVTExit|  
+----------+  
|2014-06-03|  
| null|  
|2012-10-23|  
|2014-06-03|  
|2015-11-05|

As I can see this EVTExit is converted to Date.

`**workset.write.format("org.elasticsearch.spark.sql").save("workset/workset1")**`

But after writing it to the ES, I am getting it same EPOC format.

**"EVTExit" : 1401778800000**

Can anyone have the ideas whats going wrong here.

Thanks

---

<div class="post-metadata">

**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\
**Post date:** [October 24, 2016, 6:55pm UTC](https://discuss.elastic.co/t/elasticsearch-spark/61610/2 "2016-10-24T18:55:10Z")

</div>

Did you set ES mapping correctly?

Also, try setting es.mapping.date.rich to false. ([https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html#cfg-field-info](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html#cfg-field-info))

---

<div class="post-metadata">

**Author:** ![nayan\_sharma](https://avatars.discourse-cdn.com/v4/letter/n/d6d6ee/32.png) [@nayan\_sharma](https://discuss.elastic.co/u/nayan_sharma)\
**Post date:** [October 25, 2016, 7:13am UTC](https://discuss.elastic.co/t/elasticsearch-spark/61610/3 "2016-10-25T07:13:03Z")

</div>

val df2 = df.withColumn("EVTExit\_1", $"EVTExit".cast("string"))  
df2.write.format("org.elasticsearch.spark.sql").option("es.mapping.date.rich", "false").save("workset/workset1")

ya it did the work.Thank You.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:22pm UTC](https://discuss.elastic.co/t/elasticsearch-spark/61610/4 "2017-07-06T13:22:48Z")

</div>