# How to Parse App Log from Apache nifi

**URL:** <https://discuss.elastic.co/t/how-to-parse-app-log-from-apache-nifi/95015>\
**Category:** Logstash\
**Created:** [July 28, 2017, 9:09pm UTC](https://discuss.elastic.co/t/how-to-parse-app-log-from-apache-nifi/95015 "2017-07-28T21:09:54Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![john5634](https://avatars.discourse-cdn.com/v4/letter/j/d78d45/32.png) [@john5634](https://discuss.elastic.co/u/john5634)\
**Post date:** [July 28, 2017, 9:09pm UTC](https://discuss.elastic.co/t/how-to-parse-app-log-from-apache-nifi/95015/1 "2017-07-28T21:09:54Z")

</div>

We are trying to get the first 3 fields from a text app log file

```
2017-07-28 11:40:00,256 WARN [Timer-Driven Process Thread-7] o.a.h.c.protocol.ResponseProcessCookies Invalid cookie header: "set-cookie: guest_id=v1%3A150125640009836952; Expires=Sun, 28 Jul 2019 15:40:00 UTC; Path=/; Domain=.twitter.com". Invalid 'expires' attribute: Sun, 28 Jul 2019 15:40:00 UTC
2017-07-28 11:59:46,909 INFO [Provenance Maintenance Thread-1] o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers for events starting with ID 65697100` 

```

We want the first 3 fields to be set and the rest to show up a a regular message. Here is the example of our logstash conf file.

```
input {

    tcp {
            port => 10000
    }
}
filter{

grok{
 match => {"message" => "%{WORD:date} %{WORD:time} %{WORD:EventType}"}
}

}
output {
  elasticsearch { hosts => ["localhost:9200"] }
}

```

any ideas?

Thank you!!!

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [July 28, 2017, 10:54pm UTC](https://discuss.elastic.co/t/how-to-parse-app-log-from-apache-nifi/95015/2 "2017-07-28T22:54:05Z")

</div>

You can use something like this:

```auto
%{DATE:date} %{TIME:time} %{WORD:EventType} %{GREEDYDATA:EventText}

```

You will have a field with the date in the format yy-mm-dd, a field with the time, a field with the event type and a field with the rest of the message.

LIke this:

```auto
{
  "date": [
    "17-07-28"
  ],
  "time": [
    "11:40:00,256"
  ],
  "EventType": [
    "WARN"
  ],
  "EventText": [
    "[Timer-Driven Process Thread-7] o.a.h.c.protocol.ResponseProcessCookies Invalid cookie header: "set-cookie: guest_id=v1%3A150125640009836952; Expires=Sun, 28 Jul 2019 15:40:00 UTC; Path=/; Domain=.twitter.com". Invalid 'expires' attribute: Sun, 28 Jul 2019 15:40:00 UTC"
  ]
}

```

You can test grok patterns using [grok debug](https://grokdebug.herokuapp.com/) or if you have Kibana 5.5 with X-Pack, it's a new feature.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 25, 2017, 10:54pm UTC](https://discuss.elastic.co/t/how-to-parse-app-log-from-apache-nifi/95015/3 "2017-08-25T22:54:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.