# Logstash plus filebeat pipeline

**URL:** <https://discuss.elastic.co/t/logstash-plus-filebeat-pipeline/162282>\
**Category:** Logstash\
**Created:** [December 27, 2018, 9:06pm UTC](https://discuss.elastic.co/t/logstash-plus-filebeat-pipeline/162282 "2018-12-27T21:06:31Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![fefo69](https://avatars.discourse-cdn.com/v4/letter/f/b5a626/32.png) [@fefo69](https://discuss.elastic.co/u/fefo69)\
**Post date:** [December 27, 2018, 9:06pm UTC](https://discuss.elastic.co/t/logstash-plus-filebeat-pipeline/162282/1 "2018-12-27T21:06:31Z")

</div>

Hi there,

I have a api log file with the following format

...  
2018.27.12 17:37:28.423 GET /api/v1/catalogManagement/productOffering 200 4  
2018.27.12 17:37:28.242 GET /api/v1/addressManagement/address 200 1214  
...

I would like to extract all the fields from log file and parse with grok to setup the @timestamp according first two records in each row

Thanks!

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 28, 2018, 3:07am UTC](https://discuss.elastic.co/t/logstash-plus-filebeat-pipeline/162282/2 "2018-12-28T03:07:44Z")

</div>

What have you got so far? Have you gone through [This introduction to Logstash](https://www.elastic.co/blog/a-practical-introduction-to-logstash)?

---

<div class="post-metadata">

**Author:** ![fefo69](https://avatars.discourse-cdn.com/v4/letter/f/b5a626/32.png) [@fefo69](https://discuss.elastic.co/u/fefo69)\
**Post date:** [January 4, 2019, 8:54pm UTC](https://discuss.elastic.co/t/logstash-plus-filebeat-pipeline/162282/3 "2019-01-04T20:54:50Z")

</div>

The main issue I have is how to parse the date based on dots and timestamp and setup the field timestamp

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [January 4, 2019, 9:07pm UTC](https://discuss.elastic.co/t/logstash-plus-filebeat-pipeline/162282/4 "2019-01-04T21:07:59Z")

</div>

What have you got so far? What is not working?

---

<div class="post-metadata">

**Author:** ![fefo69](https://avatars.discourse-cdn.com/v4/letter/f/b5a626/32.png) [@fefo69](https://discuss.elastic.co/u/fefo69)\
**Post date:** [January 10, 2019, 9:26pm UTC](https://discuss.elastic.co/t/logstash-plus-filebeat-pipeline/162282/5 "2019-01-10T21:26:02Z")

</div>

I need parsing dates and timestamps from fields, and then using that date and timestamp as the logstash timestamp for the event.

The fomat of hte lines in the file is like this:  
...  
2018.27.12 17:37:28.423 GET /api/v1/catalogManagement/productOffering 200 4  
...

Are you able to do it using grok ?

---

<div class="post-metadata">

**Author:** ![shyamari](https://avatars.discourse-cdn.com/v4/letter/s/8e8cbc/32.png) [@shyamari](https://discuss.elastic.co/u/shyamari)\
**Post date:** [January 17, 2019, 1:24pm UTC](https://discuss.elastic.co/t/logstash-plus-filebeat-pipeline/162282/6 "2019-01-17T13:24:01Z")

</div>

you can try patterns on the below link:

[http://grokdebug.herokuapp.com/](http://grokdebug.herokuapp.com/)

I tried matching the date with pattern:

%{YEAR}.%{MONTHDAY}.%{MONTHNUM}

---

<div class="post-metadata">

**Author:** ![fefo69](https://avatars.discourse-cdn.com/v4/letter/f/b5a626/32.png) [@fefo69](https://discuss.elastic.co/u/fefo69)\
**Post date:** [January 17, 2019, 2:40pm UTC](https://discuss.elastic.co/t/logstash-plus-filebeat-pipeline/162282/7 "2019-01-17T14:40:06Z")

</div>

Thanks so much for your input, I was able to make it work with a solution similar to the one you are proposing and I was also able to setup @timestamp using date. The app logs were quite tricky also becouse I had different date formats, now I fixed them. I was using hat debug site, its very useful

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 14, 2019, 2:40pm UTC](https://discuss.elastic.co/t/logstash-plus-filebeat-pipeline/162282/8 "2019-02-14T14:40:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.