I have rewritten the Ruby filter mentioned here:
The filter now reads as such and I am able to at minimum pass the event to OpenSearch:
input: |-
http {
port => 8080
codec => "json"
filter: |-
json {
source => "message"
target => "assets"
}
ruby {
init => '
require "net/http"
require "uri"
jira_url = ENV["JIRA_DEV_URL"] + ENV["ASSETS_ENDPOINT"]
uri = URI(jira_url)
token = ENV["JIRA_BEARER"]
request = Net::HTTP::Get.new(uri)
request["Authorization"] = "Bearer " + token
response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: uri.scheme == "https") do |http|
http.request(request)
return response
end
@@json_list = JSON.parse(response.body)
'
code => '
begin
event.set("[assets][environment]", "dev")
event.get('[assets][object][data]").each do |key, value|
should_remove = false
if value.nil?
should_remove = true
elsif (value.is_a?(String) || value.is_a?(Array) || value.is_a?(Hash)) && value.empty?
should_remove = true
end
if should_remove
event.remove("[assets][object][data][#{key}]")
else
matching_item = @@json_list.find { |item| item["id"] == key }
if matching_item
new_key = matching_item["name"].downcase.gsub(" ", "_")
event.remove("[assets][object][data][#{key}]")
event.set("[assets][object][data][#{new_key}]", value)
end
end
end
rescue Exception => e
logger.error("{e.message}")
end
'
}
output: |-
opensearch {
hosts => ["url:port"]
index => "index_string"
ssl => true
ssl_certificate_verification => true
keystore => "keystore.p12"
keystore_password => "${KEYSTORE_PASSWORD}"
truststore => "truststore.jks"
truststore_password => "${TRUSTSTORE_PASSWORD}"
manage_template => false
}
stdout {}
However I am not getting the full JSON output with the following tags added and a bunch of data and/or comments missing.
The original filter managed to retrieve object data in a big json dump but has since stopped functioning hence the rewrite.
ruby {
init => "
require 'net/http'
require 'json'
require 'openssl'
def fetch_data(uri, token)
http = Net::HTTP.new.(uri.hostname, uri.port)
http.use_ssl = true
request = Net::HTTP::Get.new(uri)
request['Authorization'] = 'Bearer ' + token
response = http.request(request)
return response
end
"
code => "
begin
objectId = event.get('[assets][objectId]')
objectId = objectId.to_s if objectId.is_a?(Integer)
webhookTrigger = event.get('[assets][event]')
token = ENV['JIRA_BEARER']
# Object Data
url = 'https://some.thing/jira/rest/assets/latest/object/'+objectId
uri = URI(url)
response = fetch_data(uri, token)
if response.code.to_i == 200
event.set('[assets][object][data]', response.body)
end
# Object Comments
url = 'https://some.thing/jira/rest/assets/latest/comment/object/'+objectId
uri = URI(url)
response = fetch_data(uri, token)
if response.code.to_i == 200
event.set('[assets][object][comments]', response.body)
end
rescue Exception => e
event.set('ruby_exception', e.message)
event.set('ruby_exception_backtrace', e.backtrace.join('\n'))
end
"
}
What am I missing from the above filter, is it an issue with the json parsing or the ruby?
)