Curl a webservice using ruby filter

Elastic Stack Logstash
1

Hello,

Im trying to consume some events from a webservice with curl using ruby code in logstash filter. The service requires an ip address to be supplied with -d switch.

curl -s -XPOST -d 'ip=151.101.0.81' 'http://api.greynoise.io:8888/v1/query/ip'

input {"ip":"151.101.0.81"} into the code snippet, it doesnt work as expected. Might be missing something in the ruby section.

input { stdin { codec => json_lines } }

filter { 

 ruby {
 init => "
   require 'net/http'
   require 'uri'
   require 'json'
 "
 code => "
   uri = URI.parse('http://api.greynoise.io:8888/v1/query/ip')
   request = Net::HTTP::Post.new(uri)
   request.set_form_data(
     "ip" => event.get('[ip]'),
   )

   response = Net::HTTP.get_response(uri)
   if response.code == '200'
     result = JSON.parse(response.body)
     event.set('[graynoise_status]', result)
   else
     event.set('[graynoise_status]', 'ERROR reaching greynoise service')
     end
    "
  }
}

output {
   stdout { codec => "rubydebug" }
 }
2

@Badger has always been super helpful in the past :slight_smile:

3

This needs to be a hash, and if you want to use double quotes you need to escape them.

request.set_form_data( { 'ip' => event.get('[ip]') } )

You need to request the request you have just built, and this does not do that.

            http = Net::HTTP.new(uri.host, uri.port)
            response = http.request(request)
4

Thank you. Was able to create the correct request using this tool - https://jhawthorn.github.io/curl-to-ruby/.

The working filter:

filter {

  ruby {
     init => "
       require 'net/http'
       require 'uri'
       require 'json'
     "
     code => "
       uri = URI.parse('http://api.greynoise.io:8888/v1/query/ip')
       request = Net::HTTP::Post.new(uri)
       request.set_form_data(
        { 'ip' => event.get('[ip]') }
       )

     req_options = {
       use_ssl: uri.scheme == 'https',
     }

     response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
       http.request(request)
     end

       if response.code == '200'
         result = JSON.parse(response.body)
         event.set('[graynoise_status]', result)
       end
     "
   }

}