Locate json field with jsonpath (logstash)

Elastic Stack Logstash
1

Hi Elastic team,

I have the next Json:

{"response-code":"4000","response":{"result":[{"DetailsPageURL":"/show.do?resourceid=22&method=show&PRINTER_FRIENDLY=true","TODAYUNAVAILPERCENT":"0","Attribute":[{"DISPLAYNAME":"Tiempo de respuesta","Value":"0","Units":" mins","AttributeID":"2202"}],"HEALTHATTRIBUTEID":"2201","TARGETADDRESS":"","RESOURCENAME":"test.sh","TODAYAVAILPERCENT":"100","TARGETNAME":"","TODAYSCHEDDOWNPERCENT":"0.0","AVAILABILITYATTRIBUTEID":"2200","HEALTHSEVERITY":"5","DISKUTIL":"-1","MANAGED":"true","PHYMEMUTIL":"-1","AVAILABILITYMESSAGE":"Resource up. <br>The resource test is available.","AVAILABILITYSEVERITY":"5","TYPESHORTNAME":"Script","TYPE":"Script Monitor","DESCRIPTION":"Script Monitor Monitoring: for test.sh","LASTPOLLEDTIME":"02-nov-2022 15:08","RESOURCEID":"22","TODAYUNMANGDPERCENT":"0.0","HEALTHMESSAGE":"Cleared by BR_Admin_wlibrev","CPUUTIL":"-1","CHILDMONITORS":[{"AVAILABILITYID":"10002766","DISPLAYNAME":"NumTrx","CHILDMONITORINFO":[{"DISPLAYNAME":"Nodo3","RESOURCEID":"30597165","CHILDATTRIBUTES":[{"DISPLAYNAME":"COUNT","Value":"2235.0","Units":" ","AttributeID":"10002801"}],"AVAILABILITYSEVERITY":"-","HEALTHSEVERITY":"-"},{"DISPLAYNAME":"Nodo4","RESOURCEID":"30597166","CHILDATTRIBUTES":[{"DISPLAYNAME":"COUNT","Value":"59.0","Units":" ","AttributeID":"10002801"}],"AVAILABILITYSEVERITY":"-","HEALTHSEVERITY":"-"},{"DISPLAYNAME":"Nodo5","RESOURCEID":"30597167","CHILDATTRIBUTES":[{"DISPLAYNAME":"COUNT","Value":"336.0","Units":" ","AttributeID":"10002801"}],"AVAILABILITYSEVERITY":"-","HEALTHSEVERITY":"-"},{"DISPLAYNAME":"Nodo6","RESOURCEID":"30597168","CHILDATTRIBUTES":[{"DISPLAYNAME":"COUNT","Value":"483.0","Units":" ","AttributeID":"10002801"}],"AVAILABILITYSEVERITY":"-","HEALTHSEVERITY":"-"},{"DISPLAYNAME":"Nodo7","RESOURCEID":"30597169","CHILDATTRIBUTES":[{"DISPLAYNAME":"COUNT","Value":"1726.0","Units":" ","AttributeID":"10002801"}],"AVAILABILITYSEVERITY":"-","HEALTHSEVERITY":"-"},{"DISPLAYNAME":"Nodo8","RESOURCEID":"30597170","CHILDATTRIBUTES":[{"DISPLAYNAME":"COUNT","Value":"401.0","Units":" ","AttributeID":"10002801"}],"AVAILABILITYSEVERITY":"-","HEALTHSEVERITY":"-"},{"DISPLAYNAME":"TotalNodos","RESOURCEID":"30597171","CHILDATTRIBUTES":[{"DISPLAYNAME":"COUNT","Value":"5240.0","Units":" ","AttributeID":"10002801"}],"AVAILABILITYSEVERITY":"-","HEALTHSEVERITY":"-"},{"DISPLAYNAME":"timestamp","RESOURCEID":"30597172","CHILDATTRIBUTES":[],"AVAILABILITYSEVERITY":"-","HEALTHSEVERITY":"-"},{"DISPLAYNAME":"timeStamp","RESOURCEID":"30890095","CHILDATTRIBUTES":[{"DISPLAYNAME":"COUNT","Value":"1509.0","Units":" ","AttributeID":"10002801"}],"AVAILABILITYSEVERITY":"-","HEALTHSEVERITY":"-"}],"HEALTHID":"10002767"}],"DISPLAYNAME":"test","HEALTHSTATUS":"clear","RCAPageURL":"/jsp/RCA.jsp?resourceid=22&attributeid=2201","IMAGEPATH":"/images/script.gif","AVAILABILITYSTATUS":"up"}],"uri":"/app/json/get"}}

I need to extract just the value into "Total Nodos" located in:

"prueba" => "%{[response][result][0][CHILDMONITORS][0][CHILDMONITORINFO][10][CHILDATTRIBUTES][0][Value]}"

if i search with this method works good, the problema is that sometimes the location of the filed change, for example from "10" with "12".

So, i need to be able to find the same value but with the name of the field into the json.

I tryed something like this:

"prueba" => "%{[response][result][0][CHILDMONITORS][0][CHILDMONITORINFO][DISPLAYNAME=="TotalNodos"][CHILDATTRIBUTES][0][Value]}"

But that sintaxis doesn't work.

I apreciate your help.

2

I would use a ruby filter to find the right entry in the array:

    ruby {
        code => '
            begin
                a = event.get("[response][result][0][CHILDMONITORS][0][CHILDMONITORINFO]")
                a = a.select { |x| x["DISPLAYNAME"] == "TotalNodos" }
                event.set("someField", a[0]["CHILDATTRIBUTES"][0]["Value"])
            rescue
            end
        '
    }
3

Great advice @Badger works perfect.

Thanks for your help.

4

Hi @Badger,

Despite the code works, i have the next error when the position of the value change:

[2022-11-15T09:30:17,191][WARN ][logstash.javapipeline    ][log_trans] Waiting for input plugin to close {:pipeline_id=>"log_trans", :thread=>"#<Thread:0x7371c607@/home/logstash/logstash-7.12.0/logstash-core/lib/logstash/pipeline_action/create.rb:54 run>"}
[2022-11-15T09:30:18,192][WARN ][logstash.javapipeline    ][log_trans] Waiting for input plugin to close {:pipeline_id=>"log_trans", :thread=>"#<Thread:0x7371c607@/home/logstash/logstash-7.12.0/logstash-core/lib/logstash/pipeline_action/create.rb:54 run>"}
[2022-11-15T09:30:19,202][ERROR][logstash.javapipeline    ][log_trans] Dropping events to unblock input plugin {:pipeline_id=>"log_trans", :count=>125, :thread=>"#<Thread:0x7371c607@/home/logstash/logstash-7.12.0/logstash-core/lib/logstash/pipeline_action/create.rb:54 run>"}
[2022-11-15T09:30:19,304][ERROR][logstash.javapipeline    ][log_trans] Dropping events to unblock input plugin {:pipeline_id=>"log_trans", :count=>1, :thread=>"#<Thread:0x7371c607@/home/logstash/logstash-7.12.0/logstash-core/lib/logstash/pipeline_action/create.rb:54 run>"}
[2022-11-15T09:30:27,750][INFO ][logstash.javapipeline    ][log_trans] Pipeline terminated {"pipeline.id"=>"log_trans"}

This error kill the pipeline and i have to restart all logstash because don't start the pipeline on its own.

5

I suggest you ask a new question about that.

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.