String interpolation for HTTP output message format


I am trying to send some data gathered with the JDBC input plugin to an HTTP endpoint. I have to make my POST body match a certain specification, and so I use message format for the HTTP output plugin.

This is the message format:

message =>  '[{"headers":{"timestamp":"%{@timestamp}","toplevel_hostgroup":"%{toplevel_hostgroup}"},"body":%{body}}]'

In the body, I have to send the whole event as a JSON, in string. To make the whole event into a string I use the ruby filter plugin:

code => "event['body'] = event.to_json"

I check the events with the stdout rubydebug codec, they look like this:

                 "relid" => 16479,
            "schemaname" => "public",
               "relname" => "latest_reports",
        "heap_blks_read" => 2565,
         "heap_blks_hit" => 33262,
         "idx_blks_read" => 3666,
          "idx_blks_hit" => 11562145,
       "toast_blks_read" => 0,
        "toast_blks_hit" => 0,
        "tidx_blks_read" => 0,
         "tidx_blks_hit" => 0,
              "@version" => "1",
            "@timestamp" => "2015-11-18T10:05:52.577Z",
    "toplevel_hostgroup" => "punch",
                  "body" => "{\"relid\":16479,\"schemaname\":\"public\",\"relname\":\"latest_reports\",\"heap_blks_read\":2565,\"heap_blks_hit\":33262,\"idx_blks_read\":3666,\"idx_blks_hit\":11562145,\"toast_blks_read\":0,\"toast_blks_hit\":0,\"tidx_blks_read\":0,\"tidx_blks_hit\":0,\"@version\":\"1\",\"@timestamp\":\"2015-11-18T10:05:52.577Z\",\"toplevel_hostgroup\":\"punch\"}"

As you can see above, the whole event is nicely written in the body field, exactly how I would need it. Unfortunately the webserver receives the POST as follows:


What I would need, and expect is this:


Somehow when LogStash::StringInterpolation.evaluate is called for message it doesn't keep he body field as a string. I tried to use

code => "event['body'] = event.to_json.inspect"

but it doesn't work either, the body field will have double backslashes, and the HTTP endpoint won't take it. Unfortunately I don't have control over the HTTP endpoint, so I have to solve this on my end, with Logstash. Any ideas how could I get my desired output?


Forgot to mention this is Logstash 2.0, with Ruby 1.8.7, on a RHEL6 machine.

I ran into this myself a while back, I never did find a solution though :frowning:

So is this a bug then? From the Project Principles: Community: If a newbie has a bad time, it's a bug.

I am having a bad time. :wink:

I think I just managed to solve it. Seems it was a user error after all. :blush: For others looking for the same problem, the correct configuration was:

ruby filter

code => "event['body'] = event.to_json.inspect"

message format

message =>  '[{"headers":{"timestamp":"%{@timestamp}","toplevel_hostgroup":"%{toplevel_hostgroup}"},"body":%{body}]'

After this I got a POST with the following body: