Loop through json arrays

Elastic Stack Logstash
1

Hi,
My JSON file looks like this :
{
"bookings": [
{
"bookingId": 553267,
"bookingRef": "VO20200108460",
"statusId": 1,
"status": "Confirmed",
"rooms": [
{
"board": "Demi pension",
"paxes": {
"adults": 2,
"infant": 0,
"children": 0
},
"quantity": 1,
"room": "DOUBLE"
},
{
"board": "Demi pension",
"paxes": {
"adults": 1,
"infant": 0,
"children": 0
},
"quantity": 1,
"room": "SINGLE"
}
],
"options": } }

How do I loop through all rooms.
my config file in logstash :

input {
http_poller {
urls => {
url => "XXX"
}
request_timeout => 60
schedule => { every => "60s" }
codec => "json"
}
}
filter {
split { field => "[bookings]" }
split { field => "[bookings][rooms]" }
mutate {
rename => {
"[bookings][bookingId]" => "bookingId"
"[bookings][status]" => "status"
"[bookings][hotelId]" => "hotelId"
"[bookings][hotelName]" => "hotelName"
"[bookings][hotelCity]" => "hotelCity"
"[bookings][hotelCountry]" => "hotelCountry"
"[bookings][arrDate]" => "arrDate"
"[bookings][depDate]" => "depDate"
"[bookings][price]" => "price"
"[bookings][currency]" => "currency"
"[bookings][purchasePrice]" => "purchasePrice"
"[bookings][partnerName]" => "partnerName"
"[bookings][partnerId]" => "partnerId"
"[bookings][firstName]" => "firstName"
"[bookings][lastName]" => "lastName"
"[bookings][channel]" => "channel"
"[bookings][supplierName]" => "supplierName"
"[bookings][rooms][board]" => "board"
"[bookings][rooms][paxes][adults]" => "adults"
"[bookings][rooms][paxes][infant]" => "infant"
"[bookings][rooms][paxes][children]" => "children"
"[bookings][rooms][quantity]" => "quantity"
"[bookings][rooms][room]" => "room"
}
remove_field => ["confirmedDate", "bookingRef", "bookings", "createdDate", "hotelAddress", "hotelPhonearrDate", "customerId", "title", "email", "city", "mobile", "supplierId","paxe", "payments", "options", "isXML"]
}
}
output {
elasticsearch {
hosts => "localhost:9200"
index => "bookings"
action => "update"
doc_as_upsert => "true"
document_id => "%{bookingId}"
}
stdout { codec => rubydebug }
}

Any help please !

2

What do you not like about the resulting output?

3

i can't have all the data :

loop
loop1085×142 24.8 KB

in the document above there is no double room

{
"board": "Demi pension",
"paxes": {
"adults": 2,
"infant": 0,
"children": 0
},
"quantity": 1,
"room": "DOUBLE"
},

4

You have a split filter that is splitting "[bookings][rooms]", so you will only have one room per document.

5

what can i do to have all the rooms per document !?

6

Remove the second split filter:

# split { field => "[bookings][rooms]" }
7

lo
lo1080×282 40.8 KB

it displays no rooms :frowning:

8

That is because you remove the bookings field. Change your mutate+rename to include

"[bookings][rooms]" => "rooms"
9

it's work..
but I can't do aggregation on the field rooms .....
i need this field rooms to build my dashboards.....

lop
lop1288×139 26.2 KB

10

OK, let's backtrack. If you go back to your original configuration, if a booking has multiple rooms, then when you do the split on [bookings][rooms] you will get multiple events with the same [booking_id]. In your elasticsearch filter you are using

document_id => "%{bookingId}"

so it will update the document by overwriting the first room with the second room. If you want multiple documents for the same booking id then remove that.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.