# Sending data from same source for different type of beat agents to logstash **URL:** \ **Category:** Logstash\ **Created:** [August 13, 2018, 7:21pm UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228 "2018-08-13T19:21:02Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)\ **Post date:** [August 13, 2018, 7:21pm UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/1 "2018-08-13T19:21:02Z")
I have got these beat agents installed on a machine : 1. winlogbeat 2. metricbeat Trying to send data to same logstash instance. Wondering, how logstash would differentiate the data from these 2 inputs and send output to elasticsearch? ---
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [August 14, 2018, 11:21am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/2 "2018-08-14T11:21:45Z")
Hi Ravi, Beats adds some fields of its own as well. Have a look at [https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-beat.html](https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-beat.html) ---
**Author:** ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)\ **Post date:** [August 14, 2018, 7:50pm UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/3 "2018-08-14T19:50:04Z")
So in the configuration for winlogbeat and metricbeat, I need to specify these? eg: for metricbeat beat.name: metricbeat in metricbeat.yml for winlogbeat beat.name: winlogbeat in winlogbeat.yml ---
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [August 15, 2018, 3:07am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/4 "2018-08-15T03:07:31Z")
Not really. Beats does it for you. Do you wish to send it to Logstash for any preprocessing? In that case you can use tags in your FileBeat config. [https://www.elastic.co/guide/en/beats/filebeat/6.2/configuration-filebeat-options.html#\_literal\_tags\_literal](https://www.elastic.co/guide/en/beats/filebeat/6.2/configuration-filebeat-options.html#_literal_tags_literal) As for metricbeat, that will send data directly to elasticsearch. ---
**Author:** ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)\ **Post date:** [August 16, 2018, 4:23pm UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/5 "2018-08-16T16:23:07Z")
Yes, I want to send both metricbeat and winlogbeat to logstash. ---
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [August 17, 2018, 10:08am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/6 "2018-08-17T10:08:24Z")
Hi Ravi, You can use the ` beat.name` field to apply conditional logic and process events in Logstash. > [@ravikt](#): > > So in the configuration for winlogbeat and metricbeat, I need to specify these? > eg: > for metricbeat > beat.name: metricbeat in metricbeat.yml > for winlogbeat > beat.name: winlogbeat in winlogbeat.yml And this is correct. You will have to specify the beat name. ---
**Author:** ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)\ **Post date:** [August 17, 2018, 11:54am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/7 "2018-08-17T11:54:49Z")
Thank you. I would also need logstash to send output to ES with different index for both beats. How the configuration for logstash would like in the .conf file for input and output? ---
**Author:** ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)\ **Post date:** [August 20, 2018, 12:55pm UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/8 "2018-08-20T12:55:39Z")
Hi Nachiket, Could you please update on the below? Thanks, Ravi ---
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [August 20, 2018, 1:04pm UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/9 "2018-08-20T13:04:16Z")
Hi Ravi, You could use the conditional to separate the two streams. ```auto if [beat][name] == "xyz" { mutate { add_field => { "indice" => "xyz" } } } else { mutate { add_field => {"indice" => "abc"} } } ``` You can then use the indice field in your elasticsearch output. ---
**Author:** ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)\ **Post date:** [August 20, 2018, 3:21pm UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/10 "2018-08-20T15:21:07Z")
Thank you Nachiket. so the logstash.conf should look like this? input { beats { port =\> 5044 } } if [beat][name] == "metricbeat" { mutate { add\_field =\> { "indice" =\> "metricbeat" } } } else { mutate { add\_field =\> {"indice" =\> "winlogbeat"} } } output { elasticsearch { hosts =\> ["172.31.1.10:9200"] index =\> "metricbeat-%{+YYYY.MM.dd}" } } output { elasticsearch { hosts =\> ["172.31.1.10:9200"] index =\> "winlogbeat-%{+YYYY.MM.dd}" } } ---
**Author:** ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)\ **Post date:** [August 20, 2018, 6:31pm UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/11 "2018-08-20T18:31:04Z")
Hi Nachiket, I am using filter after input in logstash.conf. Does this looks good? input { beats { port =\> 5044 } } filter { if [beat][name] == "metricbeat" { mutate { add\_field =\> { "indice" =\> "metricbeat"} } } else { mutate { add\_field =\> {"indice" =\> "winlogbeat"} } } } output { elasticsearch { hosts =\> ["172.31.1.10:9200"] index =\> "metricbeat-%{+YYYY.MM.dd}" } } output { elasticsearch { hosts =\> ["172.31.1.10:9200"] index =\> "winlogbeat-%{+YYYY.MM.dd}" } } Regards, Ravi ---
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [August 21, 2018, 4:37am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/12 "2018-08-21T04:37:43Z")
This config will create a single index for both filebeat and metricbeat. Is that what was intended? To create two indices, please use the `indice` variable we created in the elasticsearch output. ```auto output { elasticsearch { hosts => ["172.31.1.10:9200"] index => "%{indice}-%{+YYYY.MM.dd}" } } ``` ---
**Author:** ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)\ **Post date:** [August 21, 2018, 6:17am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/13 "2018-08-21T06:17:14Z")
No, I need to create 2 indices, one for each beat type. Here is my updated config: input { beats { port =\> 5044 } } filter { if [beat][name] == "metricbeat" { mutate { add\_field =\> { "indice" =\> "metricbeat"} } } else { mutate { add\_field =\> {"indice" =\> "winlogbeat"} } } } output { elasticsearch { hosts =\> ["172.31.1.10:9200"] index =\> "%{indice}-%{+YYYY.MM.dd}" } } output { elasticsearch { hosts =\> ["172.31.1.10:9200"] index =\> "%{indice}-%{+YYYY.MM.dd}" } } ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 21, 2018, 7:13am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/14 "2018-08-21T07:13:24Z")
> [@ravikt](#): > > output { > elasticsearch { > hosts =\> ["172.31.1.10:9200"] > index =\> "%{indice}-%{+YYYY.MM.dd}" > } As all events now contain a parameter with the index prefix, you should only have a single elasticsearch output. ---
**Author:** ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)\ **Post date:** [August 21, 2018, 9:36am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/15 "2018-08-21T09:36:47Z")
Hi Christian, I want to see separate ES output for both beat types. Is this possible? Thanks, Ravi ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 21, 2018, 9:41am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/16 "2018-08-21T09:41:43Z")
Why? If you want that you have to put conditionals around the outputs as well in order to avoid duplicates. ---
**Author:** ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)\ **Post date:** [August 21, 2018, 10:44am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/17 "2018-08-21T10:44:44Z")
I am sending different types of beat data, so would like to see ES output with different index. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 21, 2018, 10:46am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/18 "2018-08-21T10:46:39Z")
The way you have set it up now you can send data to different indices using a single output, which probably is more efficient. Why does this not work for you?? ---
**Author:** ![ravikt](https://avatars.discourse-cdn.com/v4/letter/r/dfb087/32.png) [@ravikt](https://discuss.elastic.co/u/ravikt)\ **Post date:** [August 21, 2018, 10:48am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/19 "2018-08-21T10:48:53Z")
I am ok if it can send data to different indices using a single output. Then my config should be like this? input { beats { port =\> 5044 } } filter { if [beat][name] == "metricbeat" { mutate { add\_field =\> { "indice" =\> "metricbeat"} } } else { mutate { add\_field =\> {"indice" =\> "winlogbeat"} } } } output { elasticsearch { hosts =\> ["172.31.1.10:9200"] } } ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 21, 2018, 11:01am UTC](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228/20 "2018-08-21T11:01:53Z")
> [@ravikt](#): > > output { > elasticsearch { > hosts =\> ["172.31.1.10:9200"] > index =\> "%{indice}-%{+YYYY.MM.dd}" > } > } Your output block should still look like this, but you only need one. [Next page](https://discuss.elastic.co/t/sending-data-from-same-source-for-different-type-of-beat-agents-to-logstash/144228.md?page=2)