Logstash data formatting help


(Peng Chen) #1

Hi,
I know this is very basic newbee question, but I hope can get some of your time to shed me some light on this.

Now I have structured json data after a few filters:
filter {
json {
source => "message"
}

	split {
		field => "@message"
	}
}

The data look like this:
{
"@message": {
"field name 1": "field data"
"field name 2": {
"field name 3": "field data",
"field name 4": "field data",
...
}
...
},
"@tags": [
"tag 1", "tag 2" ],
"@timestamp": "2017-08-21T23:37:47.244Z",
"@version": "1",
...
}

I wish to flattern a bit the data to something like this:
{
"field name 1": "field data",
"field name 2": "field data",
"field name 3": "field data",
"field name 4": "field data",
...
"@tags": [
"tag 1", "tag 2" ],
"@timestamp": "2017-08-21T23:37:47.244Z",
"@version": "1"
}
and take only the fields I'm interested. Would it be possible to show me how it can be achieved? Checked all the plugins but couldn't find something matching.

Thanks!


(Peng Chen) #2

Managed to solve the problem, a piece of ruby code will help:
filter {
split {
field => "@message"
}
ruby {

        code => "
            obj = event.get('@message')
			if obj.is_a?Hash
				obj.each {|k,v|
					event.set(k, v)
				}
			end
        "
    }
}

(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.