Removing square brackets from JSON output

Elastic Stack Logstash
1

Hi,

Logstash newbe here and I think it rocks.

I'm converting xml files to output JSON but I noticed that in the JSON all values that are not objects themselves are arrays. For example

{"externalID":["67"],"latitude":["52.5344444"],"longitude":["13.3519274"]}

How can I remove the array from the final output? The desired output would be

{"externalID":"67","latitude":"52.5344444","longitude":"13.3519274"}

Here is my current config file if it helps

input {
stdin {}
}
filter { 
multiline {
       pattern => "^\s\s(\s\s|\<\/property\>)"
        what => "previous"
}
xml {
        add_field => { 
        	"[partnerId]" => 11
	        "[price][average]" => ""
	        "[price][mask]" => "Day"
	    }
        store_xml => "false"
        source => "message"
        xpath => [
                "/property/id/text()", "externalID",
                "/property/landing_page_url/text()", "externalURL",
                "/property/title/text()", "name",
                "/property/type_of_property/text()", "typeOfProperty",
                "/property/city_name/text()", "city",
                "/property/zip_code/text()", "zipCode",
                "/property/country_code/text()", "countryCode",
                "/property/description/text()", "description",
                "/property/latitude/text()", "latitude",
                "/property/longitude/text()", "longitude",
                "/property/maximum_number_of_people/text()", "sleeps",
                "/property/number_of_proper_bedrooms/text()", "bedrooms",
				"/property/check_in_from/text()", "checkInFrom",
                "/property/check_out_before/text()", "checkOutBefore",
                "/property/minimum_days_of_stay/text()", "minimumStay",
                "/property/instant_booking/text()", "instantBooking",
				"/property/photos/photo/url/text()", "[photos][photo][url]",
                "/property/reviews_count/text()", "[reviews][no-of-reviews]",
                "/property/average_rating/text()", "[reviews][combined_score]",
                "/property/minimum_price_per_night/text()", "[price][minimum]",
                "/property/currency_code/text()", "[price][currency]",
                "/property/amenities/amenity/@name", "[amenities][amenity]",
                "/property/pets_not_allowed/text()", "[misc][petsAllowed]",
                "/property/non_smoking_only/text()", "[misc][non_smoking_only]",
                  "/property/children_friendly/text()", "[misc][childrenFriendly]"

        ]
}
mutate {
    remove_field => ["message", "@version", "host", "@timestamp", "path", "tags", "type" ]
    gsub => ["externalID", "\]|\[", ""]
}
}
output {
 stdout { codec => rubydebug }
 file { 
codec => "json"
path => ["/home/ec2-user/data/housetrip/results.json"]
 }
}
2 
filter {
  mutate {
    replace => ["externalID", "%{[externalID][0]}"]
  }
}

If you don't know all field names or don't want to enumerate them you can write a small Ruby snippet in a ruby filter.

1 Like
3

Works perfectly. Thanks very much