How to split into multiple events dynamically for a given json? Tried from various question in forums

Elastic Stack Logstash
10

The following may work. In a file called splitData.rb put

def register(params)
    @field = params['field']
    @target = params['target']
end

def filter(event)
    data = event.get(@field)
    event.remove(@field)
    a = []
    data.each { |x|
        e = event.clone
        e.set(@target, x)
        a << e
    }
    a
end

Then call it using

json {
    source => "message"
    remove_field => [ "message" ]
}
ruby {
    path => '/home/user/splitData.rb'
    script_params => { field => "data" target => "data" }
}

The critical point is to remove the data field before call event.clone.

It occurred to me that the split filter ought to be able to do this optimization (remove source before cloning if it is going to be overwritten). Looking at the code it appears that this line may be trying to do this. However I don't know what target refers to (not @target, which is never nil) so I am not sure what it does

5 Likes