I am using one of the field value to perform the grouping.
Field name – error_message
Since the field name have some run time error Id, I removed the numbers from it.
I used to below filter in the logstash.
filter {
mutate {
add_field => { " error_message _new" => "%{ error_message }"}
}
mutate {
gsub => [
" error_message _new", "[0-9]", ""
]
}
}
Example value in error_message :-
Data 1:-
Actual Value :- 1212 DATA_HANDLING_ERROR: Invalid Key ID ( == 0 ) -- additional information may be available in the logs
Mutated Value :- DATA_HANDLING_ERROR: Invalid Key ID ( == ) -- additional information may be available in the logs
Data 2: -
Actual Value :- 9823 DATA_NOT_FOUND: Data not created, so retry after some timeThe object was not loaded. The data was not found, which may be a normal situation (check for surrounding errors). QueryName: FIND_BY_ID Class: Reason: values: 123445566, 1
Mutated Value :- DATA_NOT_FOUND: Data not created, so retry after some timeThe object was not loaded. The data was not found, which may be a normal situation (check for surrounding errors). QueryName: FIND_BY_ID Class: Reason: values: ,
I am using Kibana to display the error message count. I am using the ‘Data Table’ visualization to display the error message count.
When I am trying to aggregate the error_message_new (mutated field), its working fine for most of the value except for few.
For Data 1 , its working fine.
For the Data 2, its not able to fetch the mutated value.
Is there any issue in the Kibana Visualization or in my filter while creating the mutated field?