# Parsing logs using Regex pattern in logstash filter

**URL:** https://discuss.elastic.co/t/parsing-logs-using-regex-pattern-in-logstash-filter/288117
**Category:** Logstash
**Created:** [November 1, 2021, 8:11am UTC](https://discuss.elastic.co/t/parsing-logs-using-regex-pattern-in-logstash-filter/288117 "2021-11-01T08:11:09Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Divyank\_Mahalle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/divyank_mahalle/32/91240_2.png) [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)
#### Post date: [November 1, 2021, 8:11am UTC](https://discuss.elastic.co/t/parsing-logs-using-regex-pattern-in-logstash-filter/288117/1 "2021-11-01T08:11:09Z")

</div>

Hi,

I am using logstash to ingest data into the Elasticsearch using a txt file.  
I build a regex query to extract msg-id field from the logs, but msg-id field is not getting extracted though the regex query is working fine.

Any changes/suggestions are required in the regex query for msg-id field.  
The original msg-id field value is "MSGID: " which is not getting mapped in output.

regex testing link-[https://regex101.com/r/R1ohvI/1](https://regex101.com/r/R1ohvI/1)

Original FIlter-  
filter-

```auto
filter{
 grok {
        break_on_match => false
        match => {
            "message" => [
                "(MSGID|Message Id)\s*:\s*(?<msgid>(?:(?!Mobile)[A-Z0-9a-z+-]){7,36})",
                ]}}}

```

Output-

```auto
{
            "host" => "dev",
         "message" => "10/11/2021 5:49:43 PM,MSGID: <CSError>Mobile_Number: <>Job_no:<>User_id<> mstrGateWay <>\r"
}

```

After adding "\<\>"sign in msg id field, I am able to get msg id field in output, but I don't want output, instead I want this output "CSError"

**Updated FIlter-**

```auto
 "message" => [
                "(MSGID|Message Id)\s*:\s*(?<msgid>(?:(?!Mobile)[A-Z0-9a-z+-<>]){7,36})",

```

Output-

```auto
 "host" => "dev",
 "message" => "10/11/2021 5:49:43 PM,MSGID: <CSError>Mobile_Number: <>Job_no:<>User_id<> mstrGateWay <>\r",
"MSGID" => "<CSError>"

```

Thank you in Adance

---

<div class="post-metadata">

### Author: ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)
#### Post date: [November 1, 2021, 3:21pm UTC](https://discuss.elastic.co/t/parsing-logs-using-regex-pattern-in-logstash-filter/288117/2 "2021-11-01T15:21:20Z")

</div>

Try this grok pattern instead:  
`%{DATE} %{TIME} (AM|PM),MSGID: <%{NOTSPACE:msgid}>Mobile_Number: <?%{NOTSPACE:mobile_number}>Job_no:<?%{NOTSPACE:user_id} mstrGateWay <?%{NOTSPACE:mst_gate_way}>\\r`

It's not perfect, and needs a bit of tweaking but if you use [https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/)  
then it will be a lot easier!

---

<div class="post-metadata">

### Author: ![Divyank\_Mahalle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/divyank_mahalle/32/91240_2.png) [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)
#### Post date: [November 12, 2021, 6:53am UTC](https://discuss.elastic.co/t/parsing-logs-using-regex-pattern-in-logstash-filter/288117/3 "2021-11-12T06:53:22Z")

</div>

Thanks @AquaX ,I will check on it,

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 10, 2021, 6:54am UTC](https://discuss.elastic.co/t/parsing-logs-using-regex-pattern-in-logstash-filter/288117/4 "2021-12-10T06:54:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.