Error while indexing list to Elastic search using pyspark

[Ravi_Ranjan]

Hi, we are facing an issue while reading a bson file as pyspark and then indexing it in ES.
The error is regarding the type of certain keys in the bson file which is an array. The error is as follows:
Data of type java.util.ArrayList cannot be used.
There is an answer to similar problem which is not helpful. The link is given below

Pyspark write list type to ES

I'm attempting to write data to elasticsearch using pyspark. I can successfully write using the "saveAsNewAPIHadoopFile" api for most cases, however, when one of the json objects is a list, it throws an error: SparkException: Data of type java.util.ArrayList cannot be used ES can ingest lists via other means, and it is a bit cumbersome to transform my list to another state. Any advice would be useful. For refference, I am using ElasticSearch 1.7.3 Spark 1.5.0-cdh5.5.1, Hadoop 2.6.0-cdh5.…

How to get rid of this?

[james.baiera]

Could you share the full stack trace?

[Ravi_Ranjan]

Solved the issue. The error was with spark not being able to handle arrayList type. So i converted all the arraylists in my data to tuple which worked.

[james.baiera]

Great to hear, cheers!