Index ORC file using Elastic

Elastic Stack Elasticsearch
1

Hi,

I am currently evaluating elastic to be our middleman to index our ORC files generated from storm bolts and use the data for real time analytics.

Since Our data-rate is huge we need a high degree of parallelism to index in near realtime.

Please let know if Elastic is capable of indexing via ORC or RC file formats.

I am reluctant to input using hive as it takes too long a time for insertion and it takes a toll on our performance.

Regards,
Asish

2

What's ORC? What's storm bolt?

3

ORC is a file format in which HIVE stores its data. It is a compressed form.

Storm - [apache storm]

BOLT is the parallel programable workflow on a worker