# Logstash grok extract error messages from nifi logs

**URL:** <https://discuss.elastic.co/t/logstash-grok-extract-error-messages-from-nifi-logs/142251>\
**Category:** Logstash\
**Created:** [July 30, 2018, 10:18pm UTC](https://discuss.elastic.co/t/logstash-grok-extract-error-messages-from-nifi-logs/142251 "2018-07-30T22:18:45Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![databeata](https://avatars.discourse-cdn.com/v4/letter/d/5e9695/32.png) [@databeata](https://discuss.elastic.co/u/databeata)\
**Post date:** [July 30, 2018, 10:18pm UTC](https://discuss.elastic.co/t/logstash-grok-extract-error-messages-from-nifi-logs/142251/1 "2018-07-30T22:18:45Z")

</div>

How do I extract error messages from nifi logs?

log 1:

2018-07-30 18:43:32,656 ERROR [Timer-Driven Process Thread-8] o.a.n.p.a.storage.ListAzureBlobStorage ListAzureBlobStorage[id=1c111111-0164-1000-ca9e-6d37fd307870] Failed to perform listing on remote host due to java.io.IOException: com.microsoft.azure.storage.StorageException: The specified container does not exist.: {}

extract 1:

com.microsoft.azure.storage.StorageException

log 2:

2018-07-30 17:22:13,282 ERROR [Timer-Driven Process Thread-8] o.a.n.p.a.storage.PutAzureBlobStorage PutAzureBlobStorage[id=ae11111b-3cfa-1cba-ce19-c38a228875a9] Failed to put Azure blob : java.lang.IllegalArgumentException: The argument must not be null or an empty string. Argument name: blobName.

extract 2:

java.lang.IllegalArgumentException

This is my grok now.

grok {  
match =\> {  
"message" =\> "%{NOTSPACE:log\_date} %{NOTSPACE:log\_time} %{NOTSPACE:log\_level} %{GREEDYDATA:log\_text}"  
}  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 31, 2018, 11:30am UTC](https://discuss.elastic.co/t/logstash-grok-extract-error-messages-from-nifi-logs/142251/2 "2018-07-31T11:30:39Z")

</div>

That looks like a very reasonable initial pattern. What do you not like about the results?

---

<div class="post-metadata">

**Author:** ![databeata](https://avatars.discourse-cdn.com/v4/letter/d/5e9695/32.png) [@databeata](https://discuss.elastic.co/u/databeata)\
**Post date:** [July 31, 2018, 2:21pm UTC](https://discuss.elastic.co/t/logstash-grok-extract-error-messages-from-nifi-logs/142251/3 "2018-07-31T14:21:13Z")

</div>

I dont get error message categories like "com.microsoft.azure.storage.StorageException", "java.lang.IllegalArgumentException" etc. in a separate field because I have {GREEDYDATA:log\_text}.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 31, 2018, 2:59pm UTC](https://discuss.elastic.co/t/logstash-grok-extract-error-messages-from-nifi-logs/142251/4 "2018-07-31T14:59:36Z")

</div>

You could pull some additional fields out if it is useful using

```
    grok { match => { "message" => ["%{NOTSPACE:log_date} %{NOTSPACE:log_time} %{NOTSPACE:log_level} \[(?<thread>[^\]]+)\] %{NOTSPACE:classname} %{WORD:class}\[(?<id>[^\]]+)\] %{GREEDYDATA:log_text}" ] } }

```

Then pick out the last thing that looks like a class name in log\_text using

```
    grok { match => { "log_text" => ["%{HOSTNAME}: %{HOSTNAME:exception}", "%{HOSTNAME:exception}"] } }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 28, 2018, 2:59pm UTC](https://discuss.elastic.co/t/logstash-grok-extract-error-messages-from-nifi-logs/142251/5 "2018-08-28T14:59:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.