# Reading elasticsearch data using spark SQL is too slow

**URL:** <https://discuss.elastic.co/t/reading-elasticsearch-data-using-spark-sql-is-too-slow/222393>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [March 6, 2020, 2:24am UTC](https://discuss.elastic.co/t/reading-elasticsearch-data-using-spark-sql-is-too-slow/222393 "2020-03-06T02:24:24Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![hyungsun\_lim](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hyungsun_lim/32/50790_2.png) [@hyungsun\_lim](https://discuss.elastic.co/u/hyungsun_lim)\
**Post date:** [March 6, 2020, 2:24am UTC](https://discuss.elastic.co/t/reading-elasticsearch-data-using-spark-sql-is-too-slow/222393/1 "2020-03-06T02:24:24Z")

</div>

Hello, i have a question about spark - es.

I write code like below

```auto
# Initializing PySpark
from pyspark import SparkContext, SparkConf, SQLContext

# Spark Config
conf = SparkConf().setAppName("es_app")
sc = SparkContext(conf=conf)

# sqlContext
sqlContext = SQLContext(sc)

# ES to dataframe
df = sqlContext.read.format("org.elasticsearch.spark.sql").option("es.nodes","xxx.xxx.xxx.xxx:9200").option("es.nodes.discovery", "true").load("sample")

# make view 
df.registerTempTable("sample")

# Too long
sqlContext.sql("SELECT count(*) from sample").show()

```

The 'sample' index contain 5,000,000 documents.

However when i query about sql.

It take so long time to get result. (20 min takes approximately)

Maybe something wrong, but i don't know the reason.

Do i have to add more option?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 3, 2020, 2:24am UTC](https://discuss.elastic.co/t/reading-elasticsearch-data-using-spark-sql-is-too-slow/222393/2 "2020-04-03T02:24:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.