# Ruby filter to check fields dynamically **URL:** \ **Category:** Logstash\ **Created:** [September 5, 2022, 6:42pm UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709 "2022-09-05T18:42:09Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Anusha\_Kusanghi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anusha_kusanghi/32/98109_2.png) [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Post date:** [September 5, 2022, 6:42pm UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709/1 "2022-09-05T18:42:09Z")
Hie All, From the following logs I want to check on condition HasErrorState = true and drop the entire event. Data field will be single entry and in that we have multiple entries for Scales. ```auto "Data": [ { "Scales": [ { "DataId": 1, "CumulsLastCalculated": "/Date(-62135557200000+0100)/", "FiscalMemoryLastUpdated": "/Date(-62135557200000+0100)/", "HasErrorState": true, "IsActive": true, "ScaleName": "KIBANA", "TaskInfos": [ { "TaskComment": "", "TaskSuffix": "" } ] } ], { "DataId": 2, "CumulsLastCalculated": "/Date(-62135557200000+0100)/", "FiscalMemoryLastUpdated": "/Date(-62135557200000+0100)/", "HasErrorState": true, "IsActive": true, "ScaleName": "KIBANA", "TaskInfos": [ { "TaskComment": "", "TaskSuffix": "" } ] } } ] ``` Tried this but its not working as expected: ruby { code =\> ' event.get([Data][0][Scales]).each do |item| if item["HasErrorState"] == "true" event.set("ScaleName", "done") end end ' } ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 5, 2022, 7:30pm UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709/2 "2022-09-05T19:30:23Z")
> [@Anusha\_Kusanghi](#): > > if item["HasErrorState"] == "true" You are comparing it to a string. I suggest you try ``` if item["HasErrorState"] == true event.cancel end ``` ---
**Author:** ![Anusha\_Kusanghi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anusha_kusanghi/32/98109_2.png) [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Post date:** [September 6, 2022, 11:54am UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709/3 "2022-09-06T11:54:46Z")
Hie @Badger , Im not sure but if I put that correctly , But in scales it is a collection of arrays , like 0 , 1 ,2 ... entries. So if the first entry has HasErrorState=true than I have to drop that document/entry to be stored in elastic. Tried the above solution but seeing a rubyexception. Is there is any other way to do this ? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 6, 2022, 3:44pm UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709/4 "2022-09-06T15:44:39Z")
Are you saying you want to delete any entries from [Data][0][Scales] which have HasErrorState true but retain the event? What to do if [Data][0][Scales] ends up as an empty array? ---
**Author:** ![Anusha\_Kusanghi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anusha_kusanghi/32/98109_2.png) [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Post date:** [September 6, 2022, 4:00pm UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709/5 "2022-09-06T16:00:09Z")
Yes @Badger ,you are right on deleting and retain part: Are you saying you want to delete any entries from [Data][0][Scales] which have HasErrorState true but retain the event - YES If the array is empty we just have to ignore the if condition part as we can't do much in that part. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 6, 2022, 5:01pm UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709/6 "2022-09-06T17:01:59Z")
Please check the JSON that you showed. [Data][0][Scales] is an array that contains a single hash, when the array ends another hash starts. That's not valid JSON. I suspect you will want something like ``` ruby { code => ' scales = event.get("[Data][0][Scales]") scales = scales.delete_if { |item| item["HasErrorState"] } event.set("[Data][0][Scales]", scales) ' } ``` ---
**Author:** ![Anusha\_Kusanghi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anusha_kusanghi/32/98109_2.png) [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Post date:** [September 6, 2022, 6:01pm UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709/7 "2022-09-06T18:01:16Z")
IT still doesn't work , May be ill try to correct my mistake in first point. The input here is: ```auto "Data": [ { "Scales": [ { "DataId": 1, "CumulsLastCalculated": "/Date(-62135557200000+0100)/", "FiscalMemoryLastUpdated": "/Date(-62135557200000+0100)/", "HasErrorState": true, "IsActive": true, "ScaleName": "KIBANA", "TaskInfos": [ { "TaskComment": "", "TaskSuffix": "" } ] }, { "DataId": 2, "CumulsLastCalculated": "/Date(-62135557200000+0100)/", "FiscalMemoryLastUpdated": "/Date(-62135557200000+0100)/", "HasErrorState": true, "IsActive": true, "ScaleName": "KIBANA", "TaskInfos": [ { "TaskComment": "", "TaskSuffix": "" } ] } ] } ] ``` So in array Scales have 2 entries where 1st entry has HasErrorState=true , Which I want to store in my Elasticsearch. And the other entry HasErrorState=flase , Is what I want to drop. @Badger , Thanks lot for helping out. Ruby is still a new thing for me so 🫣 ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 6, 2022, 6:10pm UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709/8 "2022-09-06T18:10:44Z")
If the data has that format then the ruby filter I posted should work. ---
**Author:** ![Anusha\_Kusanghi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anusha_kusanghi/32/98109_2.png) [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Post date:** [September 6, 2022, 6:22pm UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709/9 "2022-09-06T18:22:01Z")
Error I see is: [2022-09-06T20:15:11,312][ERROR][logstash.filters.ruby][main] Ruby exception occurred: undefined method `delete_if' for nil:NilClass {:class=>"NoMethodError", :backtrace=>["(ruby filter code):4:in `block in filter\_method'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:96:in `inline_script'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:89:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:159:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:178:in `block in multi\_filter'", "org/jruby/RubyArray.java:1821:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:175:in `multi\_filter'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:134:in `multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:300:in `block in start\_workers'"]} [2022-09-06T20:20:40,193][INFO][logstash.javapipeline][main] Pipeline terminated {"pipeline.id"=\>"main"} ---
**Author:** ![Anusha\_Kusanghi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anusha_kusanghi/32/98109_2.png) [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Post date:** [September 6, 2022, 6:42pm UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709/10 "2022-09-06T18:42:30Z")
@Badger This small correction in deed worked for me: ruby { code =\> ' scales = event.get("[Data][0][Scales]") scales = scales.delete\_if { |item| item["HasErrorState"] == false } event.set("[Data][0][Scales]", scales) ' } ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 4, 2022, 6:42pm UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709/11 "2022-10-04T18:42:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.