Newbee question

Elastic Stack Elasticsearch
1

Hello,
I am newbee to ES and have basic question. I have a log file that i would
like to feed into ES and get indexed for search/retrieval. The log has the
following format:

Timestamp Description: Host;type;state;status;code;detail : Message :
Detailed Message

I downloaded ES and set it up but not sure how to feed this file
dynamically into the ES and get it indexed. I appreciate any
pointers/guidance with examples.

Thanks,
Bhaskar

2

you need a log parser for you log file (that you have to write by your own
) that can extract values of each field that you mentioned in a sequential
manner and then feed these name value pairs to build the index.

On Fri, Jun 22, 2012 at 10:49 PM, Bhaskar bmarthi@gmail.com wrote:

Hello,
I am newbee to ES and have basic question. I have a log file that i would
like to feed into ES and get indexed for search/retrieval. The log has the
following format:

Timestamp Description: Host;type;state;status;code;detail : Message :
Detailed Message

I downloaded ES and set it up but not sure how to feed this file
dynamically into the ES and get it indexed. I appreciate any
pointers/guidance with examples.

Thanks,
Bhaskar

--
Saurabh Kumar
M.Sc (Mathematics) B.E (Computer Science)
Birla Institute of Technology and Science-Pilani

3

If the thing that generates logs is rsyslog, you might want to check
out omelasticsearch output module. You can find a tutorial here:
http://wiki.rsyslog.com/index.php/HOWTO:_rsyslog_%2B_elasticsearch

Any questions or feedback about that tutorial are welcome :D.
omelasticsearch is not in the stable release of rsyslog just yet, but
it will be pretty soon.

If you don't use rsyslog, or you don't like omelasticsearch for some
reason, there are other ready-made options out there. What I found
interesting I wrote here, along with some other advice on using
Elasticsearch for logs:

On 22 iun., 22:49, Saurabh Kumar saurabh.k1...@gmail.com wrote:

you need a log parser for you log file (that you have to write by your own
) that can extract values of each field that you mentioned in a sequential
manner and then feed these name value pairs to build the index.

On Fri, Jun 22, 2012 at 10:49 PM, Bhaskar bmar...@gmail.com wrote:

Hello,
I am newbee to ES and have basic question. I have a log file that i would
like to feed into ES and get indexed for search/retrieval. The log has the
following format:

Timestamp Description: Host;type;state;status;code;detail : Message :
Detailed Message

I downloaded ES and set it up but not sure how to feed this file
dynamically into the ES and get it indexed. I appreciate any
pointers/guidance with examples.

Thanks,
Bhaskar

--
Saurabh Kumar
M.Sc (Mathematics) B.E (Computer Science)
Birla Institute of Technology and Science-Pilani

4

Once you extract the required info from logs, you can use the Java APIs provided by ES for indexing
http://www.elasticsearch.org/guide/reference/java-api/index_.html

Regards,
saiyan