# Logstash not parsing metadata from logfile

**URL:** https://discuss.elastic.co/t/logstash-not-parsing-metadata-from-logfile/260496
**Category:** Logstash
**Created:** [January 8, 2021, 4:37am UTC](https://discuss.elastic.co/t/logstash-not-parsing-metadata-from-logfile/260496 "2021-01-08T04:37:04Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Rahul3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rahul3/32/77664_2.png) [@Rahul3](https://discuss.elastic.co/u/Rahul3)
#### Post date: [January 8, 2021, 4:37am UTC](https://discuss.elastic.co/t/logstash-not-parsing-metadata-from-logfile/260496/1 "2021-01-08T04:37:04Z")

</div>

```auto
Hi,
I am very new to ELK Stack, I have a requirement to parse specific metadata from a log entry,
Here is a sample log entry,

[ERROR] [2021-01-04 14:56:41,566] [http-nio-8080-exec-4] [com.blocks.bear.server.exception.ExceptionHelper.handleServerExceptions(ExceptionHelper.java:63)] - {
  "method": "getSection",
  "userId": "31369",
  "jobId": "A1706",
  "stacktrace": "javax.servlet.http.HttpServlet.service(HttpServlet.java:660): DfServiceException while logging into Docs
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.apache.soap.server.RPCRouter.invoke(RPCRouter.java:146)
	"
}

My requirement is to get below mentioned fields as individual entries,

 method: getSection
 userId: 31369
 jobId: A1706
 stacktrace: javax.servlet.http.HttpServlet.service(HttpServlet.java:660): DfServiceException while logging into Docs
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

Here is my logstash.conf file specifications 

input { 
  file {
    type => "java"
    path => "C:/ELK/test_logs.log"
    codec => multiline {
      pattern => "^%{TIMESTAMP_ISO8601} "
      negate => "true"
      what => "previous"
	}
  }
}
filter{
  grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:date}%{GREEDYDATA:misc}{\n (?m)%{GREEDYDATA:metadata}\n (?m)%{GREEDYDATA:stacktrace}"}
  }
}
output {

  stdout {
    codec => rubydebug
  }
  
  elasticsearch {
    hosts => ["localhost:9200"]
  }
}

While I try to run the logstash and update the log file,
Logstash doesn't update the new entry until I stop it and before shutting down, it processes that entry but gives a grokparsefailure.

Any help would be highly appreciated.

Thanks

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 8, 2021, 3:13pm UTC](https://discuss.elastic.co/t/logstash-not-parsing-metadata-from-logfile/260496/2 "2021-01-08T15:13:44Z")

</div>

I would parse each line with a separate pattern

```
grok {
    break_on_match => false
    match => {
        "message => [
            "method: (?<method>[^\n]*)",
            "userId: (?<userId>[^\n]*)",
            "jobId: (?<jobId>[^\n]*)",
            ...
        ]
    }
}
```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 5, 2021, 3:13pm UTC](https://discuss.elastic.co/t/logstash-not-parsing-metadata-from-logfile/260496/3 "2021-02-05T15:13:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.