# Spark SQL types are not handled through basic RDD saveToEs() calls

**URL:** <https://discuss.elastic.co/t/spark-sql-types-are-not-handled-through-basic-rdd-savetoes-calls/270458>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [April 17, 2021, 6:56pm UTC](https://discuss.elastic.co/t/spark-sql-types-are-not-handled-through-basic-rdd-savetoes-calls/270458 "2021-04-17T18:56:50Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![roypiyali92](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/roypiyali92/32/132548_2.png) [@roypiyali92](https://discuss.elastic.co/u/roypiyali92)\
**Post date:** [April 17, 2021, 6:56pm UTC](https://discuss.elastic.co/t/spark-sql-types-are-not-handled-through-basic-rdd-savetoes-calls/270458/1 "2021-04-17T18:56:50Z")

</div>

I am trying to a simple operation wherein I am running a Spark job to persist a DataFrame to ElasticSearch.  
ElasticSearch Version : 5.3.0  
Scala Version - 2.12  
Spark Version - 3.0.0  
On Trying this code, I get an error

java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;

'Person person1 = new Person();  
person1.setName("Piyali");  
person1.setColour("Blue");

```
        Person person2 = new Person();
        person2.setName("Shouvik");
        person2.setColour("Black");

        List<Person> personList = new ArrayList<Person>();
        personList.add(person1);
        personList.add(person2);

        Dataset<Row> dataFrame = sparkSession.createDataFrame(personList, Person.class);

    
        JavaEsSparkSQL.saveToEs(dataFrame, "personList/doc");'

```

Whereas on trying the below code I get this error  
org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Spark SQL types are not handled through basic RDD saveToEs() calls; typically this is a mistake(as the SQL schema will be ignored). Use 'org.elasticsearch.spark.sql' package instead  
at org.elasticsearch.spark.serialization.ScalaValueWriter.org$elasticsearch$spark$serialization$ScalaValueWriter$$doWrite(ScalaValueWriter.scala:111) ~[elasticsearch-hadoop-5.3.0.jar:5.3.0]  
at org.elasticsearch.spark.serialization.ScalaValueWriter.write(ScalaValueWriter.scala:37) ~[elasticsearch-hadoop-5.3.0.jar:5.3.0]

'Person person1 = new Person();  
person1.setName("Piyali");  
person1.setColour("Blue");

```
        Person person2 = new Person();
        person2.setName("Shouvik");
        person2.setColour("Black");

        List<Person> personList = new ArrayList<Person>();
        personList.add(person1);
        personList.add(person2);

        Dataset<Row> dataFrame = sparkSession.createDataFrame(personList, Person.class);

      
        JavaEsSpark.saveToEs(dataFrame.toJavaRDD(), "personList/doc");'

```

Can somebody please help understand where am I going wrong. I am very new to both Spark and ElasticSearch . Any help will be highly appreciated.

Thanks In Advance.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 15, 2021, 6:57pm UTC](https://discuss.elastic.co/t/spark-sql-types-are-not-handled-through-basic-rdd-savetoes-calls/270458/2 "2021-05-15T18:57:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.