Use JSON structure from input in HTTP request

Elastic Stack Logstash
1

I am trying to use JSON struct from input inside the HTTP filter request.
The part in my input looks like this:

{"elemId":"1","user":[{"uid" : 1,"first": "John","last": "Smith"},{"uid" : 2,"first": "Bob","last": "Jones"},{"uid" : 4,"first": "Anna","last": "Right"}]}

Inside Logstash script, I tried to feed it as following:

filter {
  http { 
    (...)
    body => '{"script": { (...), "params": {"par1": [%{[user]}]}}'
  }
}

which results in following in HTTP request body:

{"script": { (...) "params": {"par1": [{last=Smith, uid=1, first=John},{last=Jones, uid=2, first=Bob},{last=Right, uid=4, first=Anna}]}}

Of course, it is not proper JSON, as field names and literals are not quoted and produce the following error:

"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse"}],"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"json_parse_exception","reason":"Unexpected character ('l' (code 108)): was expecting double-quote to start field name

This unexpected l is the first letter of last=Smith from the HTTP request.
How to get this JSON from input into the string literal, but with a quotation symbols? I do not know which exact fields and how many objects will be in the input file, so I cannot hardcode it.

2

If you are starting off with a field on the event that looks like this

      "user" => [
    [0] {
          "uid" => 1,
         "last" => "Smith",
        "first" => "John"
    },
    [1] {
          "uid" => 2,
         "last" => "Jones",
        "first" => "Bob"
    },
    [2] {
          "uid" => 4,
         "last" => "Right",
        "first" => "Anna"
    }
],

then you can install the json_encode plugin and use

json_encode { source => "user" target => "[@metadata][encodedUser]" }

to convert it into

    "encodedUser" => "[{\"first\":\"John\",\"last\":\"Smith\",\"uid\":1},{\"first\":\"Bob\",\"last\":\"Jones\",\"uid\":2},{\"first\":\"Anna\",\"last\":\"Right\",\"uid\":4}]",

and then mutate that

mutate { gsub => [ "[@metadata][encodedUser]", '"', '\"' ] }

to escape the quotes and get you to

    "encodedUser" => "[{\\\"first\\\":\\\"John\\\",\\\"last\\\":\\\"Smith\\\",\\\"uid\\\":1},{\\\"first\\\":\\\"Bob\\\",\\\"last\\\":\\\"Jones\\\",\\\"uid\\\":2},{\\\"first\\\":\\\"Anna\\\",\\\"last\\\":\\\"Right\\\",\\\"uid\\\":4}]",

which I think is what you want to insert into your request body.

3

Thank you. json_encode filter was the functionality I was looking for.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.