# Parsing different lines in same log

**URL:** <https://discuss.elastic.co/t/parsing-different-lines-in-same-log/171998>\
**Category:** Logstash\
**Created:** [March 12, 2019, 4:04pm UTC](https://discuss.elastic.co/t/parsing-different-lines-in-same-log/171998 "2019-03-12T16:04:13Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![mdlapla](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mdlapla/32/41829_2.png) [@mdlapla](https://discuss.elastic.co/u/mdlapla)\
**Post date:** [March 12, 2019, 4:04pm UTC](https://discuss.elastic.co/t/parsing-different-lines-in-same-log/171998/1 "2019-03-12T16:04:13Z")

</div>

Hi, I've been trying to parse a log for a couple of days now using Grok. I made it work and then, all of a sudden, it stopped working and I don't know why.

I've been using the grok debugger but I haven't been able to parse the log correctly.

This is the log I want to parse...

```
0|parity | 2019-02-20 10:31:56 UTC Imported #1400403 0xb0cd…dae6 (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)
0|parity | 2019-02-20 10:32:00 UTC Imported #1400404 0xdb1e…54ca (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)
0|parity | 2019-02-20 10:32:01 UTC 4/25 peers 4 MiB chain 180 MiB db 0 bytes queue 11 KiB sync RPC: 0 conn, 0 req/s, 0 µs
0|parity | 2019-02-20 10:32:04 UTC Imported #1400405 0x38cd…4cfc (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)
0|parity | 2019-02-20 10:32:08 UTC Imported #1400406 0x11c2…67ab (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)
0|parity | 2019-02-20 10:32:12 UTC Imported #1400407 0xba99…10f2 (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)
0|parity | 2019-02-20 10:32:16 UTC Imported #1400408 0x13b3…9bec (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)
0|parity | 2019-02-20 10:32:20 UTC Imported #1400409 0xa213…2978 (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)
0|parity | 2019-02-20 10:32:24 UTC Imported #1400410 0xbf03…59d2 (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)
0|parity | 2019-02-20 10:32:28 UTC Imported #1400411 0x155d…6da4 (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)
0|parity | 2019-02-20 10:32:32 UTC Imported #1400412 0x537f…93b8 (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)
0|parity | 2019-02-20 10:32:36 UTC Imported #1400413 0x3ea7…b66c (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)
0|parity | 2019-02-20 10:32:36 UTC 4/25 peers 4 MiB chain 180 MiB db 0 bytes queue 11 KiB sync RPC: 0 conn, 0 req/s, 0 µs
0|parity | 2019-02-20 10:32:40 UTC Imported #1400414 0x4b35…1507 (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)
0|parity | 2019-02-20 10:32:44 UTC Imported #1400415 0x06fb…d587 (0 txs, 0.00 Mgas, 0 ms, 0.57 KiB)

```

As you can see, there's two types of lines.  
On the first ones, I want to extract the block number (after #) and the transactions (before txs).  
On the second type of line, I want to extract the connected peers (the part before the /25 peers).

So far, my logstash configuration is like this:

```
input {
  beats {
    port => 5044
  }
}

filter {
  grok {
    # block message
    match => { "message" => "#%{NUMBER:block}.*%{NUMBER:txs} txs" }
    add_tag => ["blockMessage"]
  }

  grok {
    # peer message
    match => { "message" => ".* %{NUMBER:connectedPeers}/%{NUMBER:maxPeers} peers" }
    add_tag => ["peersMessage"]
  }

  mutate {
    convert => {
      "block" => "integer"
      "txs" => "integer"
      "connectedPeers" => "integer"
      "maxPeers" => "integer"
    }
  }
}

output {
   stdout { codec => rubydebug }
}

```

I've commented the mutate part, thinking that maybe Logstash was failing trying to convert, for example, txs on a connected peers line, but it still doesn't work.

¿Should I add some kind of conditional code related to each tag? ¿Do you see any error in the grok match that's preventing the line from being parsed?

Thanks for your help!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 12, 2019, 4:54pm UTC](https://discuss.elastic.co/t/parsing-different-lines-in-same-log/171998/2 "2019-03-12T16:54:56Z")

</div>

> [@mdlapla](#):
>
> it stopped working and I don't know why.

What do you mean by "it stopped working"?

---

<div class="post-metadata">

**Author:** ![mdlapla](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mdlapla/32/41829_2.png) [@mdlapla](https://discuss.elastic.co/u/mdlapla)\
**Post date:** [March 12, 2019, 5:09pm UTC](https://discuss.elastic.co/t/parsing-different-lines-in-same-log/171998/3 "2019-03-12T17:09:29Z")

</div>

That I managed to make it parse my log, went home and found grokparsefailure in the morning.  
And then I could never make it work again.

Pretty sure it was this exact same configuration minus the mutate part.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 12, 2019, 5:33pm UTC](https://discuss.elastic.co/t/parsing-different-lines-in-same-log/171998/4 "2019-03-12T17:33:41Z")

</div>

You will always have \_grokparsefailure with that configuration because the peer message grok fails on block messages, and the block message grok fails on peer messages.

---

<div class="post-metadata">

**Author:** ![mdlapla](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mdlapla/32/41829_2.png) [@mdlapla](https://discuss.elastic.co/u/mdlapla)\
**Post date:** [March 13, 2019, 12:51pm UTC](https://discuss.elastic.co/t/parsing-different-lines-in-same-log/171998/5 "2019-03-13T12:51:30Z")

</div>

Is there a way to avoid the \_grokparsefailure?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 13, 2019, 12:55pm UTC](https://discuss.elastic.co/t/parsing-different-lines-in-same-log/171998/6 "2019-03-13T12:55:32Z")

</div>

Check to see if the line is likely to match before executing the grok

```
if [message] =~ / txs / {
    grok { ... }
} elsif [message] =~ / peers / {
    grok { ... }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 10, 2019, 12:55pm UTC](https://discuss.elastic.co/t/parsing-different-lines-in-same-log/171998/7 "2019-04-10T12:55:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
