How to store only metadata in elasticsearch and indexes in the HDFS

Huge amount of Data is residing in the hdfs . we have used hive storage handler to store the data in elastic search. Ultimate goal is to use kibana to visualize the data. Now due to storage and security concerns, WE want to keep only metadata in the elastic search and refer to the data in the hdfs, so when we query, data must be read from hdfs not from ES. Is there any such provision available in the ES, or any plugins which will come for rescue in the situation.

Right Now enitire data is sitting in the ES if we are using Hive storage handler. Any help would be appreciated.

You'd need to handle that on the client layer I think. may also help.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.