# Processing Multiple Rows From JDBC Streaming

**URL:** <https://discuss.elastic.co/t/processing-multiple-rows-from-jdbc-streaming/291012>\
**Category:** Logstash\
**Created:** [December 6, 2021, 6:56am UTC](https://discuss.elastic.co/t/processing-multiple-rows-from-jdbc-streaming/291012 "2021-12-06T06:56:48Z")\
**Posts on this page:** 4\
**Page:** 2

<div class="post-metadata">

**Author:** ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)\
**Post date:** [December 8, 2021, 3:54pm UTC](https://discuss.elastic.co/t/processing-multiple-rows-from-jdbc-streaming/291012/21 "2021-12-08T15:54:07Z")

</div>

The full pipeline config that manipulates the data is here:

```auto
    if [question] {
      ruby {
        code => '
          data = event.get("question")
          if data.is_a? Array
            newData = []
            data.each { |h|
              q = h.delete("questionid")
              event.set("question#{q}", h)
            }
          end
        '
      }
      ruby {
              code => '
                  def EmptyField(object, name, event)
                      if object
                          if object.kind_of?(Hash) and object != {}
                              object.each { |k, v| EmptyField(v, "#{name}[#{k}]", event) }
                          elsif object.kind_of?(Array) and object != []
                              object.each_index { |i|
                                  EmptyField(object[i], "#{name}[#{i}]", event)
                              }
                          else
                              if object == "nil"
                                  event.remove(name)
                              end
                          end
                      end
                  end
      
                  event.to_hash.each { |k, v|
                      EmptyField(v, "[#{k}]", event)
                  }
              '
          }
      split {
        field => "question"
      }
      mutate {
        remove_field => ["[question]" ]
      }
    }

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [December 8, 2021, 6:52pm UTC](https://discuss.elastic.co/t/processing-multiple-rows-from-jdbc-streaming/291012/22 "2021-12-08T18:52:13Z")

</div>

The first thing that the EmptyField function tests is `if object`, so `if ! object` inside that branch cannot possibly test true.

```
input { generator { count => 1 lines => ['{ "question39": { "dec": null, "question": "Sup Brah?", "bool": null, "date": null, "text": null, "int": null } }'] codec => json } }
filter {
    ruby {
        code => '
            def EmptyField(object, name, event)
                if object.kind_of?(Hash) and object != {}
                    object.each { |k, v| EmptyField(v, "#{name}[#{k}]", event) }
                elsif object.kind_of?(Array) and object != []
                    object.each_index { |i|
                        EmptyField(object[i], "#{name}[#{i}]", event)
                    }
                else
                    if ! object
                        event.remove(name)
                    end
                end
            end

            event.to_hash.each { |k, v|
                EmptyField(v, "[#{k}]", event)
            }
        '
    }
}
output { stdout { codec => rubydebug { metadata => false } } }

```

results in

```
"question39" => {
    "question" => "Sup Brah?"
}
```

---

<div class="post-metadata">

**Author:** ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)\
**Post date:** [December 8, 2021, 8:37pm UTC](https://discuss.elastic.co/t/processing-multiple-rows-from-jdbc-streaming/291012/23 "2021-12-08T20:37:51Z")

</div>

Awesome thank you. This is where my inability to understand Ruby really shines, lol. Just one of the many things to add to the list of needing to give time to learn.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 5, 2022, 8:38pm UTC](https://discuss.elastic.co/t/processing-multiple-rows-from-jdbc-streaming/291012/24 "2022-01-05T20:38:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/processing-multiple-rows-from-jdbc-streaming/291012.md?page=1)