Removing New Line Characters in http output

Elastic Stack Logstash
1

Good Day,
Is there a way to remove a New Line Characters from the http_output command when sending to a server via API. The system receiving the data has to accept the data as a single string of text and it has to be plain text. So for example the following domains 'example11.com' 'example12.com' 'test.org' need to be structured so they are sent as: 'example11.comexample12.comtest.org'

The WORKING equivalent CURL command would be as follows:

curl -X PATCH "https://api.example.com/api/v3/indicators/conditions" \
 -H "Accept: application/json, text/plain, application/json, application/json" \
 -H "Authorization: Basic <password>" \
 -H "Content-Type: text/plain" \
 -d 'example11.comexample12.comtest.org' \

My Filter and output look like the following:

 filter {
  ruby {
    code => "
      sender_domain = []

      # Get the 'objects' array from the event
      objects = event.get('objects')
      if objects
        # Iterate through each object in the 'objects' array
        objects.each do |object|
          # Check if the type is 'domain'
          type = object['type']
          if type == 'domain'
            # Extract the IP address value and confidence if the type is 'domain'
            value = object['value']
            confidence = object['source_reported_confidence']

            if value
              # Determine the risk level based on the confidence value
              risk = case confidence
                     when 1..50 then 'Low'
                     when 51..70 then 'Medium'
                     when 71..89 then 'High'
                     when 100..100 then 'Critical'
                     else 'Unknown' # Default to Unknown if confidence is outside expected range
                     end

              # Only add the domain if the risk level is 'Critical'
              if risk == 'Critical'
                # Add the IP address to the sender_ips array
                sender_domain << value
              end
            end
          end
        end
      end

      # Set the extracted values addresses to 'sender_domain' in the event
      event.set('sender_domain', sender_domain.to_json)
    "
  }

  mutate {
    remove_field => ["objects"]
    remove_field => ["@version"]
    remove_field => ["@timestamp"]
    remove_field => ["meta"]
  }
}

output {
  stdout {
    codec => rubydebug
  }

    http {
    url => "https://api.example.com/api/v3/indicators/conditions"
    http_method => "patch"
    format => "message"
    content_type => "plain/text"
    headers => {
         "authorization" => "basic <password>"
       }
    message => "%{sender_domain}"
 }
}

The result comes out as:

{
    "sender_domain" => "[\"example11.com\",\"example12.com\",\"test.org\"]"
}
2

If that is really what you want (which seems unlikely) then you could use

input { generator { count => 1 lines => [ '{"sender_domain": [ "example11.com", "example12.com", "test.org"] }' ] codec => json } }

filter {
    mutate {
        # Get rid of commas between entries
        join => { "sender_domain" => "" }
    }
}
output {
    http {
        ssl_enabled => false
        format => message
        message => "%{sender_domain}"
        url=> "http://127.1.1.1:2000/"
        http_method => post
    }
}

If you have a receiving pipeline

input { http { host => "127.1.1.1" port => 2000 } }
output { stdout {} }

then it will produce

   "message" => "example11.comexample12.comtest.org",

If you leave out the mutate then the output will .to_s the array and produce

   "message" => "example11.com,example12.com,test.org",
1 Like