# Custom Logstash pipeline configuration file **URL:** \ **Category:** Logstash\ **Created:** [October 15, 2020, 9:52pm UTC](https://discuss.elastic.co/t/custom-logstash-pipeline-configuration-file/252277 "2020-10-15T21:52:41Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Post date:** [October 15, 2020, 9:52pm UTC](https://discuss.elastic.co/t/custom-logstash-pipeline-configuration-file/252277/1 "2020-10-15T21:52:41Z")
Hi, I want to make a custom pipeline using SNMP walk plugin. I need some help in the first steps to properly understand syntax. So far I have something like this ``` filter { mutate { add_field => {"host.hostname" => "NameOfDevice"} rename => {"host" => "host.ip"} } } ``` I am adding the field of a device to the pipeline(because I don't want to query It every time and the name should not change) and I'm changing the "host" field to "host.ip" Let's say this is what I get from SNMP walk input. ``` { "iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifSpeed.6" => 100000000, "iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifSpeed.4" => 1000000000, } ``` Have can I instruct logstash to pare this log in such a way It would create the below document. > ``` > { > { > "cisco.device.interface.number" : "6", > "cisco.device.interface.speed" : 100000000 > }, > { > "cisco.device.interface.number" : "4", > "cisco.device.interface.speed" : 1000000000 > } > } > > ``` The expected dashboard output from this document should be for example any table that shows speed for every interface by given "host.ip" ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 15, 2020, 10:21pm UTC](https://discuss.elastic.co/t/custom-logstash-pipeline-configuration-file/252277/2 "2020-10-15T22:21:18Z")
I would do that in ruby ``` ruby { code => ' a = [] event.to_hash.each { |k, v| if k =~ /^iso\.org\.dod\.internet\.mgmt\.mib-2\.interfaces\.ifTable\.ifEntry\.ifSpeed\.\d+$/ matches = k.scan(/(\d+)$/) a << { "cisco.device.interface.number" => matches[0][0], "cisco.device.interface.speed" => v } end } if a != [] event.set("someField", a) end ' } ``` ---
**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Post date:** [October 15, 2020, 11:37pm UTC](https://discuss.elastic.co/t/custom-logstash-pipeline-configuration-file/252277/3 "2020-10-15T23:37:37Z")
I tried your code and this is the output I got back(I have removed the part on interface number because I will later add interface name so I don't need the number) ``` { "host.hostname" => "Hostname.domain", "iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifSpeed.1" => 0, "iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifSpeed.3" => 1000000000, "host.ip" => "10.10.10.10", "iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifSpeed.4" => 1000000000, "iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifSpeed.6" => 100000000, "iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifSpeed.7" => 100000000, "@timestamp" => 2020-10-15T23:32:56.364Z, "iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifSpeed.5" => 100000000, "type" => "snmp", "iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifSpeed.2" => 0, "iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifSpeed.8" => 0, "someField" => [ [0] { "cisco.device.interface.speed" => 0 }, [1] { "cisco.device.interface.speed" => 1000000000 }, [2] { "cisco.device.interface.speed" => 1000000000 }, [3] { "cisco.device.interface.speed" => 100000000 }, [4] { "cisco.device.interface.speed" => 100000000 }, [5] { "cisco.device.interface.speed" => 100000000 }, [6] { "cisco.device.interface.speed" => 0 }, [7] { "cisco.device.interface.speed" => 0 } ] } ``` I still see the "iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifSpeed.1" etc. fields I tried to remove them with ``` mutate { remove_field => ["%{.ifSpeed}"] } ``` after the ruby filter sequence but It does not seem to work. I can see that your output is part of "someField" array. Should in this case "someField" be cisco.device.interface and the speed itself just speed? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 15, 2020, 11:49pm UTC](https://discuss.elastic.co/t/custom-logstash-pipeline-configuration-file/252277/4 "2020-10-15T23:49:19Z")
> [@Adriann](#): > > ```auto > mutate { > remove_field => ["%{.ifSpeed}"] > } > > ``` mutate+remove\_field does not process wildcards or regexps. You could use prune+blacklist\_names, or add ``` event.remove(k) ``` to the loop. > [@](#): > > Should in this case "someField" be cisco.device.interface and the speed itself just speed? I have no insight into what final format you want the data in. I am just trying to suggest ideas of how to get there. ---
**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Post date:** [October 16, 2020, 12:17am UTC](https://discuss.elastic.co/t/custom-logstash-pipeline-configuration-file/252277/5 "2020-10-16T00:17:14Z")
> [@Badger](#): > > `event.remove(k)` event.remove(k) did the work thanks! I have tried to edit your code to achieve what I think is what I need. I have made the below change > ``` > code => ' > a = [] > event.to_hash.each { |k, v| > if k =~ /^.*?ifSpeed\.\d+$/ > matches = k.scan(/(\d+)$/) > a << { "speed" => v } > event.remove(k) > end > if k =~ /^.*?ifName\.\d+$/ > matches = k.scan(/(\d+)$/) > a << { "name" => v } > event.remove(k) > end > > } > if a != [] > event.set("cisco.device.interface", a) > end > ' > > ``` ``` [...] }, [14] { "speed" => 0 }, [15] { "name" => "Virtual254" } ], ``` But the output is not what I expect. How can I make It look like this? > [1] { > "name" =\> "Ethernet0/3", > "speed" =\> 100000000 > }, ---
**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Post date:** [October 16, 2020, 12:29am UTC](https://discuss.elastic.co/t/custom-logstash-pipeline-configuration-file/252277/6 "2020-10-16T00:29:08Z")
Thank you so much for your efforts. I get a lot from this. I got my answer in [this](https://github.com/logstash-plugins/logstash-input-snmp/issues/45) GitHub thread. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 13, 2020, 12:29am UTC](https://discuss.elastic.co/t/custom-logstash-pipeline-configuration-file/252277/7 "2020-11-13T00:29:20Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.