# Newbee question

**URL:** <https://discuss.elastic.co/t/newbee-question/8198>\
**Category:** Elasticsearch\
**Created:** [June 22, 2012, 5:19pm UTC](https://discuss.elastic.co/t/newbee-question/8198 "2012-06-22T17:19:42Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![bmarthi](https://avatars.discourse-cdn.com/v4/letter/b/46a35a/32.png) [@bmarthi](https://discuss.elastic.co/u/bmarthi)\
**Post date:** [June 22, 2012, 5:19pm UTC](https://discuss.elastic.co/t/newbee-question/8198/1 "2012-06-22T17:19:42Z")

</div>

Hello,  
I am newbee to ES and have basic question. I have a log file that i would  
like to feed into ES and get indexed for search/retrieval. The log has the  
following format:

Timestamp Description: Host;type;state;status;code;detail : Message :  
Detailed Message

I downloaded ES and set it up but not sure how to feed this file  
dynamically into the ES and get it indexed. I appreciate any  
pointers/guidance with examples.

Thanks,  
Bhaskar

---

<div class="post-metadata">

**Author:** ![Saurabh](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@Saurabh](https://discuss.elastic.co/u/Saurabh)\
**Post date:** [June 22, 2012, 7:49pm UTC](https://discuss.elastic.co/t/newbee-question/8198/2 "2012-06-22T19:49:14Z")

</div>

you need a log parser for you log file (that you have to write by your own  
) that can extract values of each field that you mentioned in a sequential  
manner and then feed these name value pairs to build the index.

On Fri, Jun 22, 2012 at 10:49 PM, Bhaskar [bmarthi@gmail.com](mailto:bmarthi@gmail.com) wrote:

> Hello,  
> I am newbee to ES and have basic question. I have a log file that i would  
> like to feed into ES and get indexed for search/retrieval. The log has the  
> following format:
> 
> Timestamp Description: Host;type;state;status;code;detail : Message :  
> Detailed Message
> 
> I downloaded ES and set it up but not sure how to feed this file  
> dynamically into the ES and get it indexed. I appreciate any  
> pointers/guidance with examples.
> 
> Thanks,  
> Bhaskar

--  
Saurabh Kumar  
M.Sc (Mathematics) B.E (Computer Science)  
Birla Institute of Technology and Science-Pilani

---

<div class="post-metadata">

**Author:** ![Radu\_Gheorghe1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radu_gheorghe1/32/2688_2.png) [@Radu\_Gheorghe1](https://discuss.elastic.co/u/Radu_Gheorghe1)\
**Post date:** [June 24, 2012, 1:32pm UTC](https://discuss.elastic.co/t/newbee-question/8198/3 "2012-06-24T13:32:31Z")

</div>

If the thing that generates logs is rsyslog, you might want to check  
out omelasticsearch output module. You can find a tutorial here:  
[http://wiki.rsyslog.com/index.php/HOWTO:\_rsyslog\_%2B\_elasticsearch](http://wiki.rsyslog.com/index.php/HOWTO:_rsyslog_%2B_elasticsearch)

Any questions or feedback about that tutorial are welcome :D.  
omelasticsearch is not in the stable release of rsyslog just yet, but  
it will be pretty soon.

If you don't use rsyslog, or you don't like omelasticsearch for some  
reason, there are other ready-made options out there. What I found  
interesting I wrote here, along with some other advice on using  
Elasticsearch for logs:

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

On 22 iun., 22:49, Saurabh Kumar [saurabh.k1...@gmail.com](mailto:saurabh.k1...@gmail.com) wrote:

> you need a log parser for you log file (that you have to write by your own  
> ) that can extract values of each field that you mentioned in a sequential  
> manner and then feed these name value pairs to build the index.
> 
> On Fri, Jun 22, 2012 at 10:49 PM, Bhaskar [bmar...@gmail.com](mailto:bmar...@gmail.com) wrote:
> 
> > Hello,  
> > I am newbee to ES and have basic question. I have a log file that i would  
> > like to feed into ES and get indexed for search/retrieval. The log has the  
> > following format:
> 
> > Timestamp Description: Host;type;state;status;code;detail : Message :  
> > Detailed Message
> 
> > I downloaded ES and set it up but not sure how to feed this file  
> > dynamically into the ES and get it indexed. I appreciate any  
> > pointers/guidance with examples.
> 
> > Thanks,  
> > Bhaskar
> 
> --  
> Saurabh Kumar  
> M.Sc (Mathematics) B.E (Computer Science)  
> Birla Institute of Technology and Science-Pilani

---

<div class="post-metadata">

**Author:** ![saiyan](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@saiyan](https://discuss.elastic.co/u/saiyan)\
**Post date:** [June 25, 2012, 6:01am UTC](https://discuss.elastic.co/t/newbee-question/8198/4 "2012-06-25T06:01:45Z")

</div>

Once you extract the required info from logs, you can use the Java APIs provided by ES for indexing  
[http://www.elasticsearch.org/guide/reference/java-api/index\_.html](http://www.elasticsearch.org/guide/reference/java-api/index_.html)

Regards,  
saiyan

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:22am UTC](https://discuss.elastic.co/t/newbee-question/8198/5 "2017-07-06T03:22:46Z")

</div>


