# Parsing different messages using different filters

**URL:** <https://discuss.elastic.co/t/parsing-different-messages-using-different-filters/83570>\
**Category:** Logstash\
**Created:** [April 25, 2017, 1:26pm UTC](https://discuss.elastic.co/t/parsing-different-messages-using-different-filters/83570 "2017-04-25T13:26:18Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ibrahimsharaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibrahimsharaf/32/17304_2.png) [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\
**Post date:** [April 25, 2017, 1:26pm UTC](https://discuss.elastic.co/t/parsing-different-messages-using-different-filters/83570/1 "2017-04-25T13:26:18Z")

</div>

Hello, I have the following log-file:

```
2017-01-01 07:53:44 [connectionpool.py] DEBUG: "POST /api/v1/crawledproducts/merchant/ariika/new_crawl_iteration/ HTTP/1.1" 200 None
2017-01-01 07:53:44 [monitor_utils.py] INFO: Crawled iteration for merchant ariika started
2017-01-01 07:53:44 [utils.py] INFO: UpdateCrawlIteration._start_crawl_iteration function took 0.127 s
2017-01-01 07:53:44 [telnet.py] DEBUG: Telnet console listening on 127.0.0.1:33357
2017-01-01 07:57:22 [statscollectors.py] INFO: Dumping Scrapy stats:
{'item_scraped_count': 22,
 'invalid_items_count': 84}

```

I want to extract the merchant name `ariika` from the second line and each of the last 2 lines `(scraped_items, invalid_items)`

I wrote a filter for each which was working fine its own, but when I added the 2 filters together it didn't work, here is my logstash.conf

```
input {
    tcp {
        port => 5000
        codec => multiline {
            pattern => "^%{TIMESTAMP_ISO8601} "
            negate => true
            what => previous
        }
    }
}

filter {
        # Merchant name
	grok{
		match => ["message", "(?<merchant_name>(?<=Crawled iteration for merchant ).*(?= started))"]
	}
}

filter{
	# Scraped items, invalid items
	grok{
		match => ["message", "'item_scraped_count': %{NUMBER:scraped_items:int}"]
	}
	grok{
		match => ["message", "'invalid_items_count': %{NUMBER:invalid_items:int}"]
	}
}

output {
	stdout {
		codec => rubydebug
	}
}

```

How can I edit my configurations to extract the needed info from the logfile?

---

<div class="post-metadata">

**Author:** ![ibrahimsharaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibrahimsharaf/32/17304_2.png) [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\
**Post date:** [April 25, 2017, 5:19pm UTC](https://discuss.elastic.co/t/parsing-different-messages-using-different-filters/83570/2 "2017-04-25T17:19:33Z")

</div>

fixed using conditions

```
filter {

	if "Dumping Scrapy stats" in [message] {

		## Scraped items, invalid items
		grok{
				match => ["message", "'item_scraped_count': %{NUMBER:scraped_items:int}"]
		}
		grok{
				match => ["message", "'invalid_items_count': %{NUMBER:invalid_items:int}"]
		}
		}

	else{
		grok{
				match => ["message", "Crawled iteration for merchant %{WORD:merchant_name} started"]
			}
		}
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 23, 2017, 5:32pm UTC](https://discuss.elastic.co/t/parsing-different-messages-using-different-filters/83570/3 "2017-05-23T17:32:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.