Distinguish json pattern and send it to specific index

Elastic Stack Logstash
1

hi everyone

I have a logstash container querying data from a mysql table called events. The mysql table contains a json field that can contain different data structures, i need to identify when an event contains certain structure and send the event to the corresponding index so that every index contains all the events that follow the same json structure.

Im using the jdbc input plugin to read data from mysql and the elasticsearch output plugin to index the events.

here is an example of the different json structures you can find in the json field of the events table (i know all the different structures that can happen)

{"id":"5320","time":"1547749347","data":"27"}

{"id":"5320","time":"1547917026","data":[{"type":"1","value":"33.30"}]}

{"time":"1563742800.0","data":[{"id":"14050","type":"1","unit":"1","value":"31.7","message":"0b01000011100110000110110011001101"}]}

I have tried some filters but i couldn't find any suitable solution for this

help me :frowning:

2

Perhaps something like

    if [data][0][message] {
        mutate { add_field => { "targetIndex" => "foo" } }
    } else if [data][0][value] {
        mutate { add_field => { "targetIndex" => "bar" } }
    } else {
        mutate { add_field => { "targetIndex" => "baz" } }
    }
1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.