I have 20000 documents in ES and I have uploaded them from Hive using Hive -ES connector. How can I dump new data from hive to ES? I want to do an incremental upload how sqoop does.
@Ashwini_Kumar I'm not aware of anything that does this out of the box for Hive, so it might require a little bit of footwork to do. Does your new data have any way to differentiate it from old data such as a create timestamp or transaction ids? If so, you could attempt to query just for the new data since your last load into ES when running the load command.