# Spark 2.0.2 read data from ES 1.7.0 using spark-20\_2.11:5.0.1 got error java.io.InvalidClassException: org.apache.spark.sql.execution.FilterExec

**URL:** <https://discuss.elastic.co/t/spark-2-0-2-read-data-from-es-1-7-0-using-spark-20-2-11-5-0-1-got-error-java-io-invalidclassexception-org-apache-spark-sql-execution-filterexec/67584>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [November 30, 2016, 7:47am UTC](https://discuss.elastic.co/t/spark-2-0-2-read-data-from-es-1-7-0-using-spark-20-2-11-5-0-1-got-error-java-io-invalidclassexception-org-apache-spark-sql-execution-filterexec/67584 "2016-11-30T07:47:32Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![giaosudau](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/giaosudau/32/13396_2.png) [@giaosudau](https://discuss.elastic.co/u/giaosudau)\
**Post date:** [November 30, 2016, 7:47am UTC](https://discuss.elastic.co/t/spark-2-0-2-read-data-from-es-1-7-0-using-spark-20-2-11-5-0-1-got-error-java-io-invalidclassexception-org-apache-spark-sql-execution-filterexec/67584/1 "2016-11-30T07:47:32Z")

</div>

Hi everyone,  
I have a problem when using Zeppelin which is using `Spark 2.0.2` is reading from `ES 1.7.0` using `> org.elasticsearch:elasticsearch-spark-20_2.11:5.0.1`  
I am using Spark to join data from 2 ES clusters  
I create a DF from the ds1.  
Schema like this.  
|-- actionType: integer  
|-- adServing: string  
|-- ageRange: string  
|-- ageYr1: long  
|-- ageYr2: long  
|-- antsPv: string  
|-- author: integer  
|-- browserId: integer  
|-- browserVerId: integer  
|-- clickAdServing: string  
|-- clickBeacon: string  
|-- coId: integer  
|-- count: integer  
|-- countryCode: string  
|-- cuid: string  
|-- devFamilyId: integer  
|-- devTypeId: integer  
|-- domainId: integer  
|-- eventIds: string  
|-- fAgeRange: string  
|-- fGender: string  
|-- gender: integer  
|-- goals: string  
|-- hasProfile: integer  
|-- imkSum: string  
|-- imkTime: string  
|-- incrBounce: integer  
|-- incrVisit: integer  
|-- intSum: string  
|-- intTime: string  
|-- ip: string  
|-- ispId: integer  
|-- keywords: string  
|-- loggedDate: timestamp  
|-- loggedHour: integer  
|-- loggedTime: timestamp  
|-- newVisit: integer  
|-- object: integer  
|-- order: string  
|-- orderStatus: integer  
|-- osFamilyId: integer  
|-- osId: integer  
|-- register: integer  
|-- resId: integer  
|-- revenue: long  
|-- section: integer  
|-- sessionId: long  
|-- siteId: long  
|-- zoneId: integer

and ds2  
root  
|-- adId: integer  
|-- ageRange: string  
|-- blackList: long  
|-- broserVerId: long  
|-- browser: string  
|-- browserId: long  
|-- browserVer: string  
|-- campaignId: long  
|-- carrierId: long  
|-- channelId: long  
|-- conAgeRange: string  
|-- coordinates: string  
|-- countryCode: string  
|-- creativeId: long  
|-- devId: long  
|-- devModel: string  
|-- gender: integer  
|-- gent: integer  
|-- his: string  
|-- hostname: string  
|-- ip: string  
|-- loggedTime: timestamp  
|-- networkId: long  
|-- os: string  
|-- osId: long  
|-- refUrlId: long  
|-- refZone: string  
|-- res: string  
|-- secondPrice: double  
|-- sectionId: long  
|-- spendCost: double  
|-- spentBuyCost: double  
|-- urlId: long  
|-- userId: long  
|-- viewToClick: integer  
|-- websiteId: integer  
|-- zoneId: long

I join 2 ds together by using sql

> select campaignId, count(b.\*) as click, count(a.actionType) as pageview, avg(timeSpent) as timesSpent  
> from insight a  
> left join clickDetail b on b.his = a.clickBeacon  
> where a.loggedTime between '2016-11-30 00:00:00.0' and '2016-12-01 00:00:00.0'  
> and siteId=5384240  
> and campaignId IN (580473,583396,583768)  
> and timeSpent \> 0  
> group by campaignId  
> order by timesSpent desc

and I got the error

> java.io.InvalidClassException: org.apache.spark.sql.execution.FilterExec; local class incompatible: stream classdesc serialVersionUID = -3682127751384284016, local class serialVersionUID = -1001413048249876169

BTW If I using 2.4.2 with spark 1.x It query ok.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 28, 2016, 7:47am UTC](https://discuss.elastic.co/t/spark-2-0-2-read-data-from-es-1-7-0-using-spark-20-2-11-5-0-1-got-error-java-io-invalidclassexception-org-apache-spark-sql-execution-filterexec/67584/2 "2016-12-28T07:47:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.