I am very new to Elastic and thinking about using it in a project.
Think about text files that contain millions of JSON lines each have different structure, nearly one thousand:
Sample Text file:
I am thinking about pushing all of these lines to Kafka from source system and then use Logstash-Elastic Hadoop and store parsed data in HDFS. What is the best way of doing this ? Does logstash support these kind of parsing operation ?