Forwarding the http input data to http output data

Elastic Stack Logstash
9

No, the printed representation of your message contains escape chars. The [message] field itself does not.

I can see two ways to do this. If you do something like

    mutate { gsub => [ "message", "\A", "[", "message", "\n", ",", "message", "\Z", "]" ] }
    json { source => "message" target => "[@metadata][bulk]" remove_field => [ "message" ] }

you will end up with

 "@metadata" => {
    "bulk" => [
        [0] {
            "index" => {
                "_index" => "journaling_insert"
            }
        },
        [1] {
            "STOREATTRIBUTE3" => "Passed Value",
                   "REMOTEIP" => "1.111.1.11",
                    "EVENTID" => "16",
            "STOREATTRIBUTE5" => "StoreDB Value",
                 "FLSECURITY" => {
                "SID" => "1111"
            },
            "STOREATTRIBUTE4" => "StoreDB Value",
                 "FLCUSTOMER" => {
                 "LASTNAME" => "the Grey",
                "FIRSTNAME" => "Gandalf"
            },
            "STOREATTRIBUTE2" => "StoreDB Value",
                   "DATETIME" => "2025-03-07T19:14:58.400",
                  "CHAINCODE" => "8971"
        },
        [2] {
            "index" => {
                "_index" => "journaling_insert"
            }
        },
        ...

You would need to split that pairwise to get each document and the name of the index to write it to. That would require ruby.

You can then move all the fields of the document up to the root, and use a sprintf reference to pick the index in the output

index => "%{[@metadata][index][_index]}"

That requires a bunch of knowledge about the format of _bulk requests, and will break for anything other than a simple index command (so no support for update, delete, create etc.)

Alternatively, you already have exactly the request body that the _bulk endpoint expects, so I would suggest returning to your original approach with an http output. How exactly to reference %{[message]} in the output I cannot say, since I do not have an HTTP endpoint handy to test.