# How to put one message to several indices by condition?

**URL:** https://discuss.elastic.co/t/how-to-put-one-message-to-several-indices-by-condition/184391
**Category:** Logstash
**Created:** [June 5, 2019, 2:44pm UTC](https://discuss.elastic.co/t/how-to-put-one-message-to-several-indices-by-condition/184391 "2019-06-05T14:44:59Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![74db36a597f21b891b3f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/74db36a597f21b891b3f/32/45367_2.png) [@74db36a597f21b891b3f](https://discuss.elastic.co/u/74db36a597f21b891b3f)
#### Post date: [June 5, 2019, 2:45pm UTC](https://discuss.elastic.co/t/how-to-put-one-message-to-several-indices-by-condition/184391/1 "2019-06-05T14:45:00Z")

</div>

Hi there.  
I have a bunch of log messages that I want to parse by several ways.  
All messages are guaranteed to fit one common grok filter so they are going to index 1.  
But i want to check this messages agains other grok filters and depending of results send the same messages to other indices too.  
I wrote processing config below but now I'm confused how to make output config.  
As I understand after processing it's still one message out but I need at least two messages for two different ES indices.

```
if [log_type] == "common_type"
{
	if "condition1" in [message]
	{
		grok
		{
			match => ["message", "%{TIMESTAMP_ISO8601:log_timestamp}%{SPACE}%{NOTSPACE}%{SPACE}%{WORD:thread}%{NOTSPACE}%{SPACE}%{NOTSPACE:username}%{SPACE}%{NOTSPACE:log_level}%{SPACE}%{NOTSPACE:main}%{SPACE}%{GREEDYDATA:_log}"]
		}
		mutate
		{
			add_field => { "log_subtype" => "common_type-sl" }				
		}
	}
	
	else if "condition2" in [message]
	{
		grok
		{
			match => ["message", "%{TIMESTAMP_ISO8601:log_timestamp}%{SPACE}%{NOTSPACE}%{SPACE}%{WORD:thread}%{NOTSPACE}%{SPACE}%{NOTSPACE:username}%{SPACE}%{NOTSPACE:log_level}%{SPACE}%{NOTSPACE:main}%{SPACE}%{GREEDYDATA:_log}"]
		}			
		mutate
		{
			add_field => { "log_subtype" => "common_type-ir" }				
		}
	}
	##common filter here
	grok
	{
		match => ["message", "%{TIMESTAMP_ISO8601:log_timestamp}%{SPACE}%{NOTSPACE}%{SPACE}%{WORD:thread}%{NOTSPACE}%{SPACE}%{NOTSPACE:username}%{SPACE}%{NOTSPACE:log_level}%{SPACE}%{NOTSPACE:main}%{SPACE}%{GREEDYDATA:_log}"]
	}
	date
	{
		match => ["log_timestamp", "YYYY-MM-dd HH:mm:ss,SSS"]
		timezone => "UTC"
		add_tag => ["tsmatch"]
	}
	mutate
	{
		remove_field => ["log_timestamp", "@message", "message","source","offset"]
	}
}
```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 5, 2019, 3:54pm UTC](https://discuss.elastic.co/t/how-to-put-one-message-to-several-indices-by-condition/184391/2 "2019-06-05T15:54:58Z")

</div>

You can use conditionals in the output section just as you can in the filter section of the configuration.

---

<div class="post-metadata">

### Author: ![74db36a597f21b891b3f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/74db36a597f21b891b3f/32/45367_2.png) [@74db36a597f21b891b3f](https://discuss.elastic.co/u/74db36a597f21b891b3f)
#### Post date: [June 5, 2019, 5:15pm UTC](https://discuss.elastic.co/t/how-to-put-one-message-to-several-indices-by-condition/184391/3 "2019-06-05T17:15:15Z")

</div>

But how to make sure that one message will be processed exaxtly two times by different grok filter?  
If I understand right, when message come to output it will be parsed by latest applied filter, won't it?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 5, 2019, 6:05pm UTC](https://discuss.elastic.co/t/how-to-put-one-message-to-several-indices-by-condition/184391/4 "2019-06-05T18:05:46Z")

</div>

If you need two copies of the event to go through differernt sets of filters before being output then write them to two different pipelines.

---

<div class="post-metadata">

### Author: ![74db36a597f21b891b3f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/74db36a597f21b891b3f/32/45367_2.png) [@74db36a597f21b891b3f](https://discuss.elastic.co/u/74db36a597f21b891b3f)
#### Post date: [June 5, 2019, 8:26pm UTC](https://discuss.elastic.co/t/how-to-put-one-message-to-several-indices-by-condition/184391/5 "2019-06-05T20:26:31Z")

</div>

I'm not sure that i got you right.  
I though that no matter how many grok filters i apply. At the end I'll get one set of fields but i need two sets of different fields from one message.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 5, 2019, 9:14pm UTC](https://discuss.elastic.co/t/how-to-put-one-message-to-several-indices-by-condition/184391/6 "2019-06-05T21:14:10Z")

</div>

Look at pipeline-to-pipeline communication. In particular the [forked path](https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html#forked-path-pattern) pattern.

---

<div class="post-metadata">

### Author: ![74db36a597f21b891b3f](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/74db36a597f21b891b3f/32/45367_2.png) [@74db36a597f21b891b3f](https://discuss.elastic.co/u/74db36a597f21b891b3f)
#### Post date: [June 6, 2019, 12:30pm UTC](https://discuss.elastic.co/t/how-to-put-one-message-to-several-indices-by-condition/184391/8 "2019-06-06T12:30:07Z")

</div>

Thank you. That is exactly what I need.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 4, 2019, 12:30pm UTC](https://discuss.elastic.co/t/how-to-put-one-message-to-several-indices-by-condition/184391/9 "2019-07-04T12:30:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
