Add_field not getting the field

Elastic Stack Logstash
1

Hello guys,

I am a newbie in logstash filtering and i having issues to get a specific field.

i want to get the sum of "Accounting-Output-Octets" that is inside the "service_data_container" field, is it possible using mutate add_field? also i would like that elastic do not round up the number, i want it to show exactly as it is, is there a way to avoid the "1E+9" ?

Below is the Json format of the log, thanks in advance!

{
"_index": "pgw-2020.02",
"_type": "doc",
"_id": "FpJBRXABJNWfzHeFZoqc",
"_version": 1,
"_score": null,
"_source": {
"service_data_container": [
{
"SGSN-Address": "xxx.xxx.xxx.xxx",
"Local-Sequence-Number": 21,
"Charging-Rule-Base-Name": "Default",
"QoS-Information": {
"QoS-Class-Identifier": 9,
"APN-Aggregate-Max-Bitrate-DL": 8192000,
"APN-Aggregate-Max-Bitrate-UL": 2048000,
"Allocation-Retention-Priority": {
"Priority-Level": 15
}
},
"Change-Condition": 7,
"Change-Time": "2020-02-14T19:27:45.000000Z",
"Time-Usage": 80,
"Time-First-Usage": "2020-02-14T19:26:25.000000Z",
"Time-Last-Usage": "2020-02-14T19:27:45.000000Z",
"Accounting-Input-Packets": 2,
"Accounting-Input-Octets": 139,
"Accounting-Output-Octets": 127,
"Accounting-Output-Packets": 1,
"Rating-Group": 1
},
{
"SGSN-Address": "201.23.189.107",
"Local-Sequence-Number": 22,
"Charging-Rule-Base-Name": "Default",
"Change-Time": "2020-02-14T19:50:46.000000Z",
"Accounting-Input-Packets": 11,
"Time-Usage": 136,
"Time-First-Usage": "2020-02-14T19:27:45.000000Z",
"Time-Last-Usage": "2020-02-14T19:40:54.000000Z",
"Accounting-Input-Octets": 664,
"Accounting-Output-Octets": 140,
"Accounting-Output-Packets": 2,
"Rating-Group": 1,
"3GPP-User-Location-Info": [
130,
39,
244,
80,
158,
164,
39,
244,
80,
1,
213,
135,
1
]
}
]
}

2

I would do that in a ruby filter. Something like

ruby {
    code => '
        container = event.get("service_data_container")
        if container.is? Array
            totalOctets = 0
            container.each { |x|
                if x["Accounting-Output-Octets"]
                    totalOctets += x["Accounting-Output-Octets"]
                end
            }
            event.set("totalOctets", totalOctets)
        end
    '
}
3

Thank you for the Reply badger!

i am a newbie to logstash and Ruby, for the code you mentioned, i got the following error:

Feb 14 20:28:21 oriontvpiasi024 logstash[8674]: [2020-02-14T20:28:21,153][ERROR][logstash.filters.ruby ][main] Ruby exception occurred: undefined method `is?' for #Hash:0x56f9c6ef

there isnt something like that?

mutate{
add_field => {"UPLINK" => "[service_data_container][Accounting-Output-Octets]"}
}

how can i get the nested information in the add_field ?

however i really need to learn ruby by the way...

thanks in advance!

4

It should be is_a? rather than is?

You could use

mutate { add_field => { "foo" => "[service_data_container][1][Accounting-Output-Octets]" } }
mutate { add_field => { "bar" => "[service_data_container][2][Accounting-Output-Octets]" } }

but you cannot add them together like that.

5

It should be is_a? rather than is?

Yep, you could use both something.is_a? Hash or something.class == Hash.

6

Hi Badger, sorry for the late response.

i tried the format you recommended:

{ add_field => { "foo" => "[service_data_container][1][Accounting-Output-Octets]" } }
mutate { add_field => { "bar" => "[service_data_container][2][Accounting-Output-Octets]" } }

But in the log still not showing the value of the fields. Is there other format?

Thanks in Advanced!