I have been using logstash to read some DB restore logs. Here is some lines of sample records.
07/08/2016 6:33:22.50: START restore database
SQL2540W Restore is successful, however a warning "2539" was encountered
during Database Restore while processing in No Interrupt mode.
07/08/2016 6:33:28.93: END restore database
SQL4406W The DB2 Administration Server was started successfully.
07/08/2016 6:35:35.29: END restart server
connect reset
DB20000I The SQL command completed successfully.
07/08/2016 6:35:38.48: END p:\s6\source\system\CMD\res_uw.cmd
Here is the filter part of my conf file.
if ([message] =~ /Backup successful/){
grok{
match => {"message" => ['%{GREEDYDATA:Message}'] }
}
mutate {
add_tag => "send_to_es"
add_field => {"Timestamp" => "%{GREEDYDATA:DATETIME}"}
}
}
if ([message] =~ /warning "2539"/){
grok{
match => {"message" => ['%{GREEDYDATA:Message}'] }
}
mutate {
add_tag => "send_to_es"
add_field => {"Timestamp" => "%{GREEDYDATA:DATETIME}"}
}
}
if ([message] =~ /(END p:|END P:)/){
grok{
match => {"message" => ['%{GREEDYDATA:DATETIME}:%{SPACE}END%{SPACE}%{GREEDYDATA:Mis}'] }
remove_field => "%{GREEDYDATA:Mis}"
}
mutate {
add_tag => "send_to_es"
}
}
I want to add the data "DATETIME" extracted from the last line of my record to message to other message to index at the same time. However, it could not add the field successfully. The output will become
"message": "SQL2540W Restore is successful, however a warning \"2539\" was encountered \r\r",
"@version": "1",
"@timestamp": "2016-07-12T02:28:52.337Z",
"path": "C:/CIGNA/hkiapp67_db_restore/res_uw.log",
"host": "SIMSPad",
"type": "txt",
"Message": "SQL2540W Restore is successful, however a warning \"2539\" was encountered \r\r",
"Timestamp": "%{GREEDYDATA:DATETIME}",
"tags": [
"send_to_es"
]
How could I solve this?