# IPAM integration **URL:** \ **Category:** Kibana\ **Created:** [January 23, 2017, 8:05pm UTC](https://discuss.elastic.co/t/ipam-integration/72532 "2017-01-23T20:05:55Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![PandKing](https://avatars.discourse-cdn.com/v4/letter/p/258eb7/32.png) [@PandKing](https://discuss.elastic.co/u/PandKing)\ **Post date:** [January 23, 2017, 8:05pm UTC](https://discuss.elastic.co/t/ipam-integration/72532/1 "2017-01-23T20:05:55Z")
Hi, Is there anyone out there combining an IPAM with Kibana, ES or logstash ? I'm looking for a way to group syslog-messages and maybe beats data based on host or subnet. We have one or more vlans for an application in an IPAM which I can talk to over a rest api. There are 1175 subnets each with a description VLAN tag, netmask etc. For example out of IPAM I can symlink the subnet to it's description on my syslog server. DRAC\_Management -\> /opt/syslog-ng/logs/192\_168\_255\_44 ASA\_Management -\> /opt/syslog-ng/logs/192\_168\_254\_84 VMware\_management -\> /opt/syslog-ng/logs/192\_168\_255\_0 OpenStack\_management -\> /opt/syslog-ng/logs/192\_168\_255\_4 Apache\_roll\_A09\_2 -\> /opt/syslog-ng/logs/192\_168\_255\_8 Product\_X\_web -\> /opt/syslog-ng/logs/192\_168\_255\_12 What's the best way to create these groupings ? ---
**Author:** ![kopacko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kopacko/32/10733_2.png) [@kopacko](https://discuss.elastic.co/u/kopacko)\ **Post date:** [January 23, 2017, 11:23pm UTC](https://discuss.elastic.co/t/ipam-integration/72532/2 "2017-01-23T23:23:38Z")
I create my own tags to identify pipeline inputs. For example, all my routers and switches coming on their own dedicated Logstash input. I apply a tag giving it a pipeline name. I do the same for ESX hosts, firewalls, Web servers, Windows desktop/servers, Wireless controllers, etc. Currently, I have 22 dedicated inputs created. They all get output into the same index in Elasticsearch. But, with those unique tags applied at input time, I can track the life of the message and it gives me lots of options for searching. ---
**Author:** ![PandKing](https://avatars.discourse-cdn.com/v4/letter/p/258eb7/32.png) [@PandKing](https://discuss.elastic.co/u/PandKing)\ **Post date:** [January 24, 2017, 2:11am UTC](https://discuss.elastic.co/t/ipam-integration/72532/3 "2017-01-24T02:11:01Z")
Sounds ok for a small enterprise but I have +5000 servers and even more service instances like apache that could be sending data. Looking for a programmatic way to handle this without setting up a separate pipeline config for every tag. ---
**Author:** ![kopacko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kopacko/32/10733_2.png) [@kopacko](https://discuss.elastic.co/u/kopacko)\ **Post date:** [January 24, 2017, 10:48pm UTC](https://discuss.elastic.co/t/ipam-integration/72532/4 "2017-01-24T22:48:26Z")
I understand and can see where you are coming from. Mine is very programmatic. Each location has a standardized syslog-ng server that does all my heavy lifting for syslog messages and each location has a central Logstash target for beats and other miscellaneous inputs I have the same configs on every server, except for the local significant configuration items (subnets, city name, etc). I am also not suggesting creating a pipeline for every tag. Do it categorically. All my network switches and routers have one, all the ESX hosts have one, etc. On the front end, the more diverse you are, the more you have on the back end later. ---
**Author:** ![PandKing](https://avatars.discourse-cdn.com/v4/letter/p/258eb7/32.png) [@PandKing](https://discuss.elastic.co/u/PandKing)\ **Post date:** [January 25, 2017, 1:14am UTC](https://discuss.elastic.co/t/ipam-integration/72532/5 "2017-01-25T01:14:12Z")
That makes a little more sense let me echo what I think you said and how I would apply it to see if we're on the same page. Scope: 10,000+ log sources, 30-50k EPS. Syslog-ng is able to split and direct data to different destinations based on programatic rule. Have a separate log destination for every high level group, say there are 6,000 unique Product+Instances. Thats a logstash farm in each DC that needs to support 10,000 or more unique port destinations. Redis-\>Logstash farm could then receive each separate stream and apply additional tags and load to ES. Redis used in data critical high traffic requirements. Programmable with chef but seems a bit daunting. I guess what I was looking for is a backdoor mechanism to link the two documents in ES. For example an asset in asset\_index running an os on a network\_index running producdX\_index is logging data. Where the asset, and network, and product data also existed in ES indexes. The root of the question is how we link data related to the same system, but multiple sources. It seems the answer needs to be at the point of ingestion but has some scale concerns. ---
**Author:** ![kopacko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kopacko/32/10733_2.png) [@kopacko](https://discuss.elastic.co/u/kopacko)\ **Post date:** [January 25, 2017, 3:31am UTC](https://discuss.elastic.co/t/ipam-integration/72532/6 "2017-01-25T03:31:28Z")
Yeap, now I see what you are referring to. I had a similar issue with multiple sources from one target. We have Windows servers with Winlogbeat on them, as well as some applications that do their sysloging, and finally some filebeats for "imporant" log files. I don't have 5000 plus, like you. I am more in the range of 900-1000. In the end, the single device, is sending out to three different receivers. In my case, I've loaded configure the winlogbeat and filebeat to use the same index as my Logstash index. So I have 3 points of ingestion for that one machine, each with it's own pipeline for processing but all end up in the same index for dashboarding, reporting, etc. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 22, 2017, 3:31am UTC](https://discuss.elastic.co/t/ipam-integration/72532/7 "2017-02-22T03:31:38Z")
Closed on February 22, 2017, 3:31am UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.