# Logstash receive from multiple source & data analysis **URL:** \ **Category:** Logstash\ **Created:** [December 4, 2015, 6:09pm UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406 "2015-12-04T18:09:52Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![Karl\_Trasschaert](https://avatars.discourse-cdn.com/v4/letter/k/f9ae1b/32.png) [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Post date:** [December 4, 2015, 6:09pm UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/1 "2015-12-04T18:09:52Z")
Hi, I'm new to logstash, i've set up the "sexilog" appliance ( it's a elk appliance mostly developped for vmware logs) I'm trying to catch a extreme networks wifi controller logs. Actually, all the log are received trough the same default input file from sexilog. 1. I would like to add a tag to the logs from the wifi controller. to filter only these logs. What's the best way to do it ? Actually this is teh default input file > input { > ``` > udp { > port => 514 > type => esxi > } > > ``` > } Can i copy that file and add the source ip ? 1. This is the kind of logs i receive: > \<182\>events: EventType[State Change] MAC[EC:35:86:XX:XX:XX] IP[172.XX.X.XX] SSID[SSIDNAME] BSSID[D8:84:XX:XX:XX:XX] I suppose i have to use grok to format it correctly, what would be the righ way to do it ? Thanks in advance for yor help ---
**Author:** ![Jackal9301](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jackal9301/32/5748_2.png) [@Jackal9301](https://discuss.elastic.co/u/Jackal9301)\ **Post date:** [December 4, 2015, 6:34pm UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/2 "2015-12-04T18:34:17Z")
If that is your only input on 514 then you just choose the type field. If you have multiple inputs each input can have a type. What do you mean by source ip? Just write a grok to match what you are going to get. That is the correct way ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 4, 2015, 6:53pm UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/3 "2015-12-04T18:53:38Z")
1. Not sure exactly what you mean, but the udp input will create a field with the IP address and (I think) source port of the connecting client. If the IP address is fixed and you want to add a human-readable field or tag you can use a conditional filter similar to this: ```auto if [clientip] == "1.2.3.4" { mutate { add_tag => ["some-appliance"] } } ``` 1. Okay, so you basically have a list of key/value pairs (on the form `KEY[VALUE]`). In most key/value cases one would use the [kv filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html), but that won't work in this case so we'll have to use grok. The following can be used to parse a single such key/value pair: ```auto EventType\[(?[^\]]+)\] ``` ---
**Author:** ![Karl\_Trasschaert](https://avatars.discourse-cdn.com/v4/letter/k/f9ae1b/32.png) [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Post date:** [December 6, 2015, 12:29pm UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/4 "2015-12-06T12:29:27Z")
HI, Thanks you for your answers. Could you confirm that the "if" has to be set in the input file ? Will try the grok ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 6, 2015, 1:00pm UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/5 "2015-12-06T13:00:34Z")
Which configuration file you place the if conditional in doesn't matter. All configuration files are essentially concatenated. ---
**Author:** ![Karl\_Trasschaert](https://avatars.discourse-cdn.com/v4/letter/k/f9ae1b/32.png) [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Post date:** [December 7, 2015, 9:39am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/6 "2015-12-07T09:39:21Z")
hi, i created a filter file in the logstash conf.d foledr with that : > filter { > if [clientip] == "10.9.8.xx" { > mutate { > add\_tag =\> ["ewc"] > type =\> ["ewc"] > } > } > } But it keep the esx type . Do i need to place that in another place ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 7, 2015, 9:53am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/7 "2015-12-07T09:53:03Z")
Do you have a `clientip` field in your messages? I only gave you an example of what a conditional could look like. You'll have to adapt it to the set of fields you have at your disposal. Secondly, it's not clear what you mean by this: ```auto mutate { add_tag => ["ewc"] type => ["ewc"] } ``` Was the intention to set `type` equal to "ewc"? Or apply the `add_tag` option unless if `type` already was equal to "ewc"? Either way it won't work but if I know what you tried to do I can explain what you should do instead. ---
**Author:** ![Karl\_Trasschaert](https://avatars.discourse-cdn.com/v4/letter/k/f9ae1b/32.png) [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Post date:** [December 7, 2015, 9:58am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/8 "2015-12-07T09:58:35Z")
I d'ont have a client ip in the log but i have a host. I change it to this : > if [host] == "10.9.8.xx" { > mutate { > add\_tag =\> "ewc" > type =\> "ewc" > }} I want to change the type, because the default type is esxi. I just wan t to be able to differenciate the logs from ewc and the logs from esxi, to eassylly filter on the dashboard. It's more clean if the ewc logs have the ewc type right ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 7, 2015, 10:02am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/9 "2015-12-07T10:02:18Z")
> I want to change the type, because the default type is esxi. `type => "ewc"` for setting the type will only work for inputs. To change the type in a filter you need this: ```auto mutate { replace => { "type" => "ewc" } } ``` > It's more clean if the ewc logs have the ewc type right ? Yes, I agree. ---
**Author:** ![Karl\_Trasschaert](https://avatars.discourse-cdn.com/v4/letter/k/f9ae1b/32.png) [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Post date:** [December 7, 2015, 10:16am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/10 "2015-12-07T10:16:24Z")
yeah that works ๐Ÿ™‚ will now try to use grok. This is what i have in the message: > 182\>events: EventType[State Change] MAC[88:30:8A:7D:XX:XX IP[172.15.X.XX] SSID[SSID] BSSID[D8:84:66:3C:XX:XX] I would like to have a column eventType, on mac, one ip etc.. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 7, 2015, 10:19am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/11 "2015-12-07T10:19:48Z")
Have you looked at the suggestion for how to parse those fields using grok that I gave you last week? ---
**Author:** ![Karl\_Trasschaert](https://avatars.discourse-cdn.com/v4/letter/k/f9ae1b/32.png) [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Post date:** [December 7, 2015, 10:33am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/12 "2015-12-07T10:33:41Z")
> [@magnusbaeck](#): > > EventType[(?\[^]]+)] I'm just start to have a look on that, but i'm not really sure to hunderstand what to do with it ๐Ÿ™‚ i just test it in the grok debugger, and looks to privde the right info. What will be the correct syntax in the filter ? something like this ? > grok { > match =\> [ > "message", "EventType[(?[^]]+)]" > "message", "MAC[(?[^]]+)] > ]} ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 7, 2015, 10:45am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/13 "2015-12-07T10:45:00Z")
I suggest you cram everything into a single expression. ```auto grok { match => ["message", "... events: EventType\[(?[^]]+)\] MAC\[(?[^]]+)\] ..."] } ``` ---
**Author:** ![Karl\_Trasschaert](https://avatars.discourse-cdn.com/v4/letter/k/f9ae1b/32.png) [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Post date:** [December 7, 2015, 10:52am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/14 "2015-12-07T10:52:41Z")
Yeah that's works ๐Ÿ™‚ ---
**Author:** ![Karl\_Trasschaert](https://avatars.discourse-cdn.com/v4/letter/k/f9ae1b/32.png) [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Post date:** [December 7, 2015, 11:13am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/15 "2015-12-07T11:13:29Z")
I have different event, do i have to add a second expression ? > events: EventType[State Change] MAC[4C:7C:5F:C7:4B:89] IP[10.9.X.XXX] SSID[SSID] BSSID[D8:84:66:3C:XX:XX] > grok { > match =\> ["message", "... events: EventType[(?[^]]+)] MAC[(?[^]]+)] ..."] > match =\> ["message", "... events: EventType[(?[^]]+)] MAC[(?[^]]+)] IP[(?[^]]+)]..."] > } ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 7, 2015, 11:27am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/16 "2015-12-07T11:27:19Z")
I think what you've proposed works but [the documentation](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#plugins-filters-grok-match) suggests this instead: ```auto grok { match => { "message" => [ "pattern1", "pattern2", ..., "patternN", ] } } ``` ---
**Author:** ![Karl\_Trasschaert](https://avatars.discourse-cdn.com/v4/letter/k/f9ae1b/32.png) [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Post date:** [December 7, 2015, 11:37am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/17 "2015-12-07T11:37:06Z")
ok will try this. i still have a question , in the log i have the Detail part > EventType[Roam] MAC[4C:8D:79:4B:XX:XX] AP[APNAME2-1] FromAP[APNAME2-1] BSSID[D8:84:66:3C:XX:XX] Details: Inside AC from AP/Radio[2] to AP/Radio[1] VNS[SSID] How can i split that one : Details: Inside AC from AP/Radio[2] to AP/Radio[1] > ... events: EventType[(?[^]]+)] MAC[(?[^]]+)] ... Details[(? > > > **** > > > > [^]]+)]" Thx ---
**Author:** ![Karl\_Trasschaert](https://avatars.discourse-cdn.com/v4/letter/k/f9ae1b/32.png) [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Post date:** [December 7, 2015, 2:20pm UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/19 "2015-12-07T14:20:56Z")
HI, i think i've set up all rule now, elso for the details, don't know if it's the right way to do it. > BSSID[(?[^]]+)] Details: Inside AC from AP/Radio[(?[^]]+)] to AP/Radio[(?[^]]+)] VNS[(?[^]]+)] The last log i have is this one, i have no idea how to manage it : > events: Access Point AP Serial Number 15281595085E0000: 12/07/15 13:10:24: DCS Measured Noise -87dBm Dropped below Threshold of -80dBm on chann. 5200Mhz 3 By the way Magnus Bรคck, thank you very much for your help. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:19am UTC](https://discuss.elastic.co/t/logstash-receive-from-multiple-source-data-analysis/36406/20 "2017-07-06T05:19:42Z")