Error while indexing list to Elastic search using pyspark

Elastic Stack Elasticsearch
1

Hi, we are facing an issue while reading a bson file as pyspark and then indexing it in ES.
The error is regarding the type of certain keys in the bson file which is an array. The error is as follows:
Data of type java.util.ArrayList cannot be used.
There is an answer to similar problem which is not helpful. The link is given below

How to get rid of this?

2

Could you share the full stack trace?

3

Solved the issue. The error was with spark not being able to handle arrayList type. So i converted all the arraylists in my data to tuple which worked.

4

Great to hear, cheers!

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.