I have the following log file
Encryption: no
Accurate: no
Volume name(s): FP-Sem_Bacula_0129
Volume Session Id: 2
Volume Session Time: 1542369027
Last Volume Bytes: 324,650,643 (324.6 MB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: -+-++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: LOG DE BACKUP DO BACULA
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: =====================
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 175M 2018-11-09 05:00 /backup/bacula/mysql/bacula-2018-11-09.sql
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 201M 2018-11-11 07:54 /backup/bacula/mysql/bacula-2018-11-11.sql
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 201M 2018-11-16 11:40 /backup/bacula/mysql/bacula-2018-11-16.sql
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 28K 2018-10-16 17:28 /backup/bacula/bacula-dir.conf
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 1,0K 2014-10-30 08:15 /backup/bacula/bacula-fd.conf
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 8,0K 2013-07-29 10:35 /backup/bacula/bacula-sd.conf
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 512 2014-10-30 08:15 /backup/bacula/bconsole.conf
every line is a new event that's good!
But I need only events in my elasticsearch similar to the line below:
Nov-16 11:40 FPBackup-dir JobId 16622: AfterJob: 512 2014-10-30 08:15 /backup/bacula/bconsole.conf
my filter looks like this:
filter {
if [type] == "bacula-log" {
if "AfterJob: " not in [message] {drop { }}
}
}
But there are events I need to drop, as an example:
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: - + - + - + - + - + + + + + + + + + + + + + + + + + + + - + - + - + - + -
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: BACULA BACKUP LOG
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: =====================
I modified the Filter, thus::
filter {
if [type] == "bacula-log" {if "AfterJob: " not in [message] {drop { }} grok { match => { "message" => "%{GREEDYDATA:field1} %{GREEDYDATA:field2}\: %{GREEDYDATA:field3}"} } if [ field3 ] !~ /^\d+/ { drop { }}
}
}
But this condition does not work. Can anybody help me?
if [ field3 ] !~ /^\d+/ { drop { }}