Hello,
I am trying to send some data gathered with the JDBC input plugin to an HTTP endpoint. I have to make my POST body match a certain specification, and so I use message format for the HTTP output plugin.
This is the message format:
message => '[{"headers":{"timestamp":"%{@timestamp}","toplevel_hostgroup":"%{toplevel_hostgroup}"},"body":%{body}}]'
In the body, I have to send the whole event as a JSON, in string. To make the whole event into a string I use the ruby filter plugin:
code => "event['body'] = event.to_json"
I check the events with the stdout rubydebug codec, they look like this:
{
"relid" => 16479,
"schemaname" => "public",
"relname" => "latest_reports",
"heap_blks_read" => 2565,
"heap_blks_hit" => 33262,
"idx_blks_read" => 3666,
"idx_blks_hit" => 11562145,
"toast_blks_read" => 0,
"toast_blks_hit" => 0,
"tidx_blks_read" => 0,
"tidx_blks_hit" => 0,
"@version" => "1",
"@timestamp" => "2015-11-18T10:05:52.577Z",
"toplevel_hostgroup" => "punch",
"body" => "{\"relid\":16479,\"schemaname\":\"public\",\"relname\":\"latest_reports\",\"heap_blks_read\":2565,\"heap_blks_hit\":33262,\"idx_blks_read\":3666,\"idx_blks_hit\":11562145,\"toast_blks_read\":0,\"toast_blks_hit\":0,\"tidx_blks_read\":0,\"tidx_blks_hit\":0,\"@version\":\"1\",\"@timestamp\":\"2015-11-18T10:05:52.577Z\",\"toplevel_hostgroup\":\"punch\"}"
}
As you can see above, the whole event is nicely written in the body field, exactly how I would need it. Unfortunately the webserver receives the POST as follows:
[{"headers":{"timestamp":"2015-11-18T10:05:52.577Z","toplevel_hostgroup":"punch"},"body":{"relid":16479,"schemaname":"public","relname":"latest_reports","heap_blks_read":2565,"heap_blks_hit":33262,"idx_blks_read":3666,"idx_blks_hit":11562145,"toast_blks_read":0,"toast_blks_hit":0,"tidx_blks_read":0,"tidx_blks_hit":0,"@version":"1","@timestamp":"2015-11-18T10:05:52.577Z","toplevel_hostgroup":"punch"}}]
What I would need, and expect is this:
[{"headers":{"timestamp":"2015-11-18T10:05:52.577Z","toplevel_hostgroup":"punch"},"body":"{\"relid\":16479,\"schemaname\":\"public\",\"relname\":\"latest_reports\",\"heap_blks_read\":2565,\"heap_blks_hit\":33262,\"idx_blks_read\":3666,\"idx_blks_hit\":11562145,\"toast_blks_read\":0,\"toast_blks_hit\":0,\"tidx_blks_read\":0,\"tidx_blks_hit\":0,\"@version\":\"1\",\"@timestamp\":\"2015-11-18T10:05:52.577Z\",\"toplevel_hostgroup\":\"punch\"}"}]
Somehow when LogStash::StringInterpolation.evaluate is called for message it doesn't keep he body field as a string. I tried to use
code => "event['body'] = event.to_json.inspect"
but it doesn't work either, the body field will have double backslashes, and the HTTP endpoint won't take it. Unfortunately I don't have control over the HTTP endpoint, so I have to solve this on my end, with Logstash. Any ideas how could I get my desired output?
Cheers,
Akos
For others looking for the same problem, the correct configuration was: