JSON response parsing at attribute level

Hi,

my logstash configuration for moving JSON response isinput { beats { port => 5044 } }
filter {mutate {
split => { "message" => "@$@" }}
mutate { add_field => { "response" => "%{[message][7]}" "channel_typ" => "%{[message][2]}"}}
mutate {remove_field => [ "message","@version" ]}}output { elasticsearch { hosts => ["http://10.201.30.181:9200"] index => "hotelhub2" } }Output in kibana like thisWe want to parse attribute value into separate columns dynamically

If it is valid JSON then use a json filter

json { source => "response" }

Now I have one more challenging my data source is with multiple data format json And xml so how do I add multiple filter?

You could just pass the response to both the xml and json filters and let one fail.

You could test the first character of the response.

if [response] =~ /^{/ {
    json { ... }
} else {
    xml { ... }
}

or if there is some other field that tells you what format the response is in (e.g. a mime type somewhere) then test that.

1 Like

after pasrsing json there array value like this

image

Each value should move to new rows how do we do that the way above spread sheet showing under expected

Below is my conf

input { beats { port => 5044 } }
filter {

mutate {
split => { "message" => "@$@" }}
mutate { add_field => { "response" => "%{[message][7]}" "channel_typ" => "%{[message][2]}"}}
mutate {remove_field => [ "message","@version" ]}

json {
source => "response"
}
mutate {remove_field => [ "response" ]}

  }

output { elasticsearch { hosts => ["http://10.201.30.181:9200"] index => "hotelhub2" } }

Can you show as either JSON or rubydebug the current structure of the data as well as the structure that you want it to have?

Already added image above please refer that current result and expected result

A screenshot of a spreadsheet tells me nothing about the data structure.

{
"meta":{},
"result":[
{
"cvc_required":false,
"checkin":"2020-07-25",
"cc_required":true,
"policies":,
"checkout":"2020-07-26",
"block":[
{
"block_id":"9825107_125413879_1_42_0",
"name":"Standard Double Room - single occupancy - Non-refundable",
"payment_terms":{
"cancellation_description":"Please note, if cancelled, modified or in case of no-show, the total price of the reservation will be charged.",
"name":"Non-refundable – Prepayment needed",
"prepayment_description":"You will be charged a prepayment of the total price at any time."
},
"free_wifi":false,
"room_description":"Rooms are 30 square metres.",
"room_surface_in_feet2":322.92,
"refundable":false,
"breakfast_included":false,
"min_price":{
"currency":"USD",
"price":100.26,
"extra_charges_breakdown":{
"extra_charge":[
{
"amount":5.0,
"excluded":false,
"name":"Towel charge",
"type":"SERVICECHARGE",
"charge_amount":5.0,
"charge_price_mode":3,
"currency":"USD"

Above one is my JSON all the details need to show in kibana as table format instead of JSON format as bellow in the new index

cvc_required checkin block_id
FALSE 25-07-20 9825107_125413879_1_42_0
FALSE 25-07-20 9825107_125413879_1_42_0
FALSE 25-07-20 9825107_125413879_1_42_0
FALSE 25-07-20 9825107_125413879_1_42_0

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.