I have 20000 documents in ES and I have uploaded them from Hive using Hive -ES connector. How can I dump new data from hive to ES? I want to do an incremental upload how sqoop does.
@Ashwini_Kumar I'm not aware of anything that does this out of the box for Hive, so it might require a little bit of footwork to do. Does your new data have any way to differentiate it from old data such as a create timestamp or transaction ids? If so, you could attempt to query just for the new data since your last load into ES when running the load command.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.