I am wondering what tools I can use for streaming decoded data from binary files to elastic the fastest way.
Here is what I am trying to do.
files are located at directory .
Those files are binary encoded files to save space. After decoding each file I get human readable structs (in c++). A lot of regular c++ structs.
I want my directory to be continuously checked for new files automatically and then I will decode the new files with my own decoder code into structs . every struct object is a record in elastic.
I have exists folder updated with lots of new binary files all the time. I can’t change their format or replace it . And the format is wrriten in a very complicated way in c++ headers and there r some technicians to save space
So I have to read the data with the c++ format (the way it was wrriten)
I can’t translate it with java because it was wrriten in c++ sizes.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.