Logstash / Protobuf plugin, removing oneOf fields that aren't set (are null)

Elastic Stack Logstash
1

I'm using the logstash protobuf plugin as a mechanism to read protobuf from kafka, convert it from protobuf and write it to elasticsearch. The problem we are having is, the oneOf fields, despite being null/not set in the incoming protobuf payload, the plugin seems to set the field and assign value, for floats, a default value of 0.0. This highly undesired. We would like to the ingestion to not include any oneOf fields that are null/not set.

I've tried a few different things such as, in the pipeline .config:

    if event.get('myField_oneof').nil? 
              event.remove('myField') 
    end

That doesn't work, it's always nil for some reason.

Neither does this:

    if event.get('myField').nil? 
              event.remove('myField') 
    end

By the time this ruby code is ran, the value is already set to 0.0.

Any ideas?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.