# How to remove dynamical field? **URL:** \ **Category:** Logstash\ **Created:** [April 8, 2016, 7:33am UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772 "2016-04-08T07:33:23Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![stefansaye](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Post date:** [April 8, 2016, 7:33am UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/1 "2016-04-08T07:33:23Z")
i have got the log looks like this below. { "message" =\> "#, @varbind\_list=[#, #\>, #, #\>], @specific\_trap=1, @source\_ip="10.10.10.13", @agent\_addr=#, @generic\_trap=6\>", "host" =\> "10.10.10.13", "@version" =\> "1", "@timestamp" =\> "2016-04-08T06:22:17.711Z", "type" =\> "snmptrap", "BGP4-MIB::bgpPeerLastError\_10\_10\_10\_54" =\> "\x00\x00", "BGP4-MIB::bgpPeerState\_10\_10\_10\_54" =\> "3", "RFC1155-SMI::enterprises\_9\_9\_187\_1\_2\_1\_1\_7\_10\_10\_10\_54" =\> "", "RFC1155-SMI::enterprises\_9\_9\_187\_1\_2\_1\_1\_8\_10\_10\_10\_54" =\> "1", "source\_ip" =\> "10.10.10.13" } { "message" =\> "#, @varbind\_list=[#\>], @specific\_trap=2, @source\_ip="10.10.10.12", @agent\_addr=#, @generic\_trap=6\>", "host" =\> "10.10.10.12", "@version" =\> "1", "@timestamp" =\> "2016-04-08T07:09:13.040Z", "type" =\> "snmptrap", "MSDP-MIB::msdpPeerState\_10\_10\_10\_54" =\> "1", "source\_ip" =\> "10.10.10.12" } } i only want to get "message" 、"host" 、 "@version"、 "@timestamp" 、"type" and "source\_ip" , how to remove the other field ? Can somebody give me a hint how I can fix the problem? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 8, 2016, 7:40am UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/2 "2016-04-08T07:40:35Z")
Have a look at the prune filter. ---
**Author:** ![stefansaye](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Post date:** [April 8, 2016, 9:05am UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/4 "2016-04-08T09:05:47Z")
thank you for your help ! but if i want to add new field for the six field name and add a new field for the six field value (below type field) , for example from my log , log \_ 1 , six\_field:BGP4-MIB::bgpPeerLastError\_10\_10\_10\_54 six\_field\_value : \x00\x00 log\_2, six\_field:MSDP-MIB::msdpPeerState\_10\_10\_10\_54 six\_field\_value : 1 ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 8, 2016, 9:41am UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/5 "2016-04-08T09:41:58Z")
Sorry, I don't understand the question. Perhaps you can give an example of an input event and the desired output event. ---
**Author:** ![stefansaye](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Post date:** [April 8, 2016, 11:23pm UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/6 "2016-04-08T23:23:35Z")
> [@stefansaye](#): > > { "message" =\> "#, @varbind\_list=[#, #\>, #, #\>], @specific\_trap=1, @source\_ip="10.10.10.13", @agent\_addr=#, @generic\_trap=6\>","host" =\> "10.10.10.13","@version" =\> "1","@timestamp" =\> "2016-04-08T06:22:17.711Z","type" =\> "snmptrap","BGP4-MIB::bgpPeerLastError\_10\_10\_10\_54" =\> "\x00\x00","BGP4-MIB::bgpPeerState\_10\_10\_10\_54" =\> "3","RFC1155-SMI::enterprises\_9\_9\_187\_1\_2\_1\_1\_7\_10\_10\_10\_54" =\> "","RFC1155-SMI::enterprises\_9\_9\_187\_1\_2\_1\_1\_8\_10\_10\_10\_54" =\> "1","source\_ip" =\> "10.10.10.13"} Sorry, I didn't say it clearly.. i can use prune to get "message" 、"host" 、 "@version"、 "@timestamp" 、"type" field , but i want to want to get the other dynamical field, for example , { "message" =\> "#, @varbind\_list=[#, #\>, #, #\>], @specific\_trap=1, @source\_ip="10.10.10.13", @agent\_addr=#, @generic\_trap=6\>", "host" =\> "10.10.10.13", "@version" =\> "1", "@timestamp" =\> "2016-04-08T06:22:17.711Z", "type" =\> "snmptrap", "BGP4-MIB::bgpPeerLastError\_10\_10\_10\_54" =\> "\x00\x00", "BGP4-MIB::bgpPeerState\_10\_10\_10\_54" =\> "3", "RFC1155-SMI::enterprises\_9\_9\_187\_1\_2\_1\_1\_7\_10\_10\_10\_54" =\> "", "RFC1155-SMI::enterprises\_9\_9\_187\_1\_2\_1\_1\_8\_10\_10\_10\_54" =\> "1", "source\_ip" =\> "10.10.10.13" } i want to get the output looks like below , message :[...] timestamp:[....] host:[....] version:[...] type:[.....] BGP4-MIB::bgpPeerLastError :\x00\x00 but the line "BGP4-MIB::bgpPeerLastError :\x00\x00 " is dynamical ,so can you please tell me how to get the BGP4-MIB::bgpPeerLastError :\x00\x00 ? please help. Thanks. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 10, 2016, 8:14pm UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/7 "2016-04-10T20:14:57Z")
Oh, I see. I think you need to use a ruby filter for this. This might do it but probably isn't very efficient: ```auto ruby { code => " prefixes = ['BGP4-MIB::bgpPeerLastError', 'BGP4-MIB::bgpPeerState'] event.to_hash.each_pair { |k, v| prefixes.each { |p| if k.start_with? p event[p] = v event.remove(k) end } } " } ``` ---
**Author:** ![stefansaye](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Post date:** [April 11, 2016, 1:36am UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/8 "2016-04-11T01:36:06Z")
> [@stefansaye](#): > > "message" =\> "#, @varbind\_list=[#\>], @specific\_trap=2, @source\_ip="10.10.10.12", @agent\_addr=#, @generic\_trap=6\>","host" =\> "10.10.10.12","@version" =\> "1","@timestamp" =\> "2016-04-08T07:09:13.040Z","type" =\> "snmptrap","MSDP-MIB::msdpPeerState\_10\_10\_10\_54" =\> "1","source\_ip" =\> "10.10.10.12" i am sorry i think i didn't say it clearly. the line "BGP4-MIB::bgpPeerLastError :\x00\x00 " is not always have the same prefix , for example , when i get the log as follows, { "message" =\> "#, @varbind\_list=[#, #\>, #, #\>], @specific\_trap=1, @source\_ip="10.10.10.13", @agent\_addr=#, @generic\_trap=6\>", "host" =\> "10.10.10.13", "@version" =\> "1", "@timestamp" =\> "2016-04-08T06:22:17.711Z", "type" =\> "snmptrap", "RFC1155-SMI::enterprises\_9\_9\_187\_1\_2\_5\_1\_17\_32\_1\_14\_16\_255\_255\_17\_0\_0\_0\_0\_0\_0\_0\_0\_2" =\> "\x00\x00", "RFC1155-SMI::enterprises\_9\_9\_187\_1\_2\_5\_1\_3\_32\_1\_14\_16\_255\_255\_17\_0\_0\_0\_0\_0\_0\_0\_0\_2" =\> "3", "RFC1155-SMI::enterprises\_9\_9\_187\_1\_2\_5\_1\_28\_32\_1\_14\_16\_255\_255\_17\_0\_0\_0\_0\_0\_0\_0\_0\_2" =\> "", "RFC1155-SMI::enterprises\_9\_9\_187\_1\_2\_5\_1\_29\_32\_1\_14\_16\_255\_255\_17\_0\_0\_0\_0\_0\_0\_0\_0\_2" =\> "1", "source\_ip" =\> "10.10.10.13" } then i want to get the output looks like below , message :[...] timestamp:[....] host:[....] version:[...] type:[.....] RFC1155-SMI::enterprises :\x00\x00 and for example 2 , when i get the log as follows, { "message" =\> "#, @varbind\_list=[#, #\>, #, #\>], @specific\_trap=1, @source\_ip="10.10.10.13", @agent\_addr=#, @generic\_trap=6\>", "host" =\> "10.10.10.13", "@version" =\> "1", "@timestamp" =\> "2016-04-08T06:22:17.711Z", "type" =\> "snmptrap", "MSDP-MIB::msdpPeerState\_10\_10\_11\_90" =\> "1", "source\_ip" =\> "10.10.10.13" } then i want to get the output looks like below , message :[...] timestamp:[....] host:[....] version:[...] type:[.....] MSDP-MIB::msdpPeerState\_10\_10\_11\_90 :1 i think maybe it's can use prefix to filter , because I neither know the field names, nor the number of fields. Do you know other method? I truly appreciate ... your help in resolving the problem. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 11, 2016, 5:39am UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/9 "2016-04-11T05:39:21Z")
As long as the set of interesting prefixes is known the example I gave should work for your second example. For the first example, how is Logstash supposed to choose between the four fields with the common prefix? ---
**Author:** ![stefansaye](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Post date:** [April 11, 2016, 6:13am UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/10 "2016-04-11T06:13:14Z")
> [@stefansaye](#): > > RFC1155-SMI::enterprises\_9\_9\_187\_1\_2\_5\_1\_17\_32\_1\_14\_16\_255\_255\_17\_0\_0\_0\_0\_0\_0\_0\_0\_2" =\> "\x00\x00", for the first example, should get as follows, RFC1155-SMI::enterprises" =\> "\x00\x00" but when i use the ruby code ,i got the error Ruby exception occurred: can't add a new key into hash during iteration {:level=\>:error} ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 11, 2016, 6:21am UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/11 "2016-04-11T06:21:43Z")
> for the first example, > should get as follows, > RFC1155-SMI::enterprises" =\> "\x00\x00" I understand that, but how should Logstash reach that conclusion? Since there are multiple fields with that prefix how is Logstash supposed to know which one to pick? > Ruby exception occurred: can't add a new key into hash during iteration {:level=\>:error} Change `event.to_hash.each_pair` to `event.to_hash.clone.each_pair`. ---
**Author:** ![stefansaye](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Post date:** [April 11, 2016, 6:32am UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/12 "2016-04-11T06:32:34Z")
Count from the message beginning of the number of sixth . but i don't know how to make logstash to do ? for example, { "message" =\> "..", "host" =\> "10.10.10.13", "@version" =\> "1", "@timestamp" =\> "2016-04-08T06:22:17.711Z", "type" =\> "snmptrap", "MSDP-MIB::msdpPeerState\_10\_10\_11\_90" =\> "1", "field\_7"=\>"field\_7 content", "source\_ip" =\> "10.10.10.13" } i want to pick the sixth field is "MSDP-MIB::msdpPeerState\_10\_10\_11\_90" =\> "1", i want the output looks like as below, "message" =\> "..", "host" =\> "10.10.10.13", "@version" =\> "1", "@timestamp" =\> "2016-04-08T06:22:17.711Z", "type" =\> "snmptrap", "MSDP-MIB::msdpPeerState\_10\_10\_11\_90" =\> "1", "source\_ip" =\> "10.10.10.13" ---
**Author:** ![stefansaye](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Post date:** [April 12, 2016, 2:34pm UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/13 "2016-04-12T14:34:25Z")
Hello, Anybody has any good suggestion? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:02am UTC](https://discuss.elastic.co/t/how-to-remove-dynamical-field/46772/14 "2017-07-06T05:02:46Z")