I'm using the parent child relationship in elastic search and I believe I need to call _refresh before bulk loading the child documents.
I currently have this code that works fine with a small test dataset, however when processing the real data set it's not loading the child documents. Is there a way to call the _refresh method via EsSpark?
EsSpark.saveToEsWithMeta( parentRDD, "myindex/parent")
EsSpark.saveToEsWithMeta( childRDD, "myindex/child")
You can index child documents without having any parents present. You'll only see a change in search results in this case as they won't have parents yet, but the parents can be added at any time afterward.
There is a configuration property that controls how the
_refresh endpoint is called during the job. It's on by default which means that the connector should be calling that end point for you already.
It's possible that something else is amiss?