Hello,
I have logs that have a similar format.
INFO - 98765 - http_Method - POST
WARN - 12345 - latency - 55
ERROR - 12345 - status_code - 200
WARN - 98765 - latency - 99
XYZ - 12345 - country - XYZ
ERROR - 98765 - status_code - 404
INFO - 12345 - http_Method - GET
XYZ - 98765 - country - ABC
in short, logs may come in any sequence. we only have four fields to care about.
We need to copy and paste data from the different fields to the logs file which contains the country field.
Expected Output:
12345-http_Method-GET-country-XYZ-status_code-200-latency-55
98765-http_Method-POST-country-ABC-status_code-404-latency-99
We are using 3 aggregate functions.
First aggregate function.
if [level] == "INFO" {
aggregate {
task_id => "%{api_id}"
code => "map['status_code'] ||= 0 ; map['status_code'] += event.get('status_code')"
}
if [level] == "XYZ" {
aggregate {
task_id => "%{api_id}"
code => "event.set('status_code', map['status_code'])"
}
Second aggregate function
if [level] == "WARN " {
aggregate {
task_id => "%{api_id}"
code => "map['latency'] ||= 0 ; map['latency'] += event.get('latency')"
}
if [level] == "XYZ" {
aggregate {
task_id => "%{api_id}"
code => "event.set('latency', map['latency'])"
}
Third aggregate function
if [level] == "ERROR " {
aggregate {
task_id => "%{api_id}"
code => "map['status_code'] ||= 0 ; map['status_code'] += event.get('status_code')"
}
if [level] == "XYZ" {
aggregate {
task_id => "%{api_id}"
code => "event.set('status_code', map['status_code'])"
}
The issue is that in the output some times, all values are present, but many times all three values are not present.
Anyone has any idea or any kind of approach please let us know.