Condition does not work (don't drop the event)


(Jamson Atila) #1

I have the following log file

Encryption: no
Accurate: no
Volume name(s): FP-Sem_Bacula_0129
Volume Session Id: 2
Volume Session Time: 1542369027
Last Volume Bytes: 324,650,643 (324.6 MB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: -+-++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: LOG DE BACKUP DO BACULA
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: =====================
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 175M 2018-11-09 05:00 /backup/bacula/mysql/bacula-2018-11-09.sql
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 201M 2018-11-11 07:54 /backup/bacula/mysql/bacula-2018-11-11.sql
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 201M 2018-11-16 11:40 /backup/bacula/mysql/bacula-2018-11-16.sql
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 28K 2018-10-16 17:28 /backup/bacula/bacula-dir.conf
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 1,0K 2014-10-30 08:15 /backup/bacula/bacula-fd.conf
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 8,0K 2013-07-29 10:35 /backup/bacula/bacula-sd.conf
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: 512 2014-10-30 08:15 /backup/bacula/bconsole.conf

every line is a new event that's good!
But I need only events in my elasticsearch similar to the line below:

Nov-16 11:40 FPBackup-dir JobId 16622: AfterJob: 512 2014-10-30 08:15 /backup/bacula/bconsole.conf

my filter looks like this:

filter {
if [type] == "bacula-log" {
if "AfterJob: " not in [message] {drop { }}
}
}

But there are events I need to drop, as an example:

16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: - + - + - + - + - + + + + + + + + + + + + + + + + + + + - + - + - + - + -
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: BACULA BACKUP LOG
16-Nov 11:40 FPBackup-dir JobId 16622: AfterJob: =====================

I modified the Filter, thus::

filter {
if [type] == "bacula-log" {

    if "AfterJob: " not in [message] {drop { }}

    grok {
    match => { "message" => "%{GREEDYDATA:field1} %{GREEDYDATA:field2}\: %{GREEDYDATA:field3}"}
    }

    if [ field3 ] !~ /^\d+/ { drop { }}

}
}

But this condition does not work. Can anybody help me?
if [ field3 ] !~ /^\d+/ { drop { }}


(Jamson Atila) #2

After many attempts I got

See my .conf file:

input {
file {
path => [ "/opt/teste.log" ]
type => "bacula-log"
start_position => beginning
sincedb_path => "/dev/null"
}
}

filter {
if [type] == "bacula-log" {

    if ([message] !~ "AfterJob\: \d{1}") { drop { } }

    grok {
    match => { "message" => "%{GREEDYDATA:campo1} %{GREEDYDATA:campo2}\: %{GREEDYDATA:campo3}"}
    }

}
}

output {
if [type] == "bacula-log" {
elasticsearch {
hosts => [ "10.0.0.224:9200" ]
index => "bacula-log-%{+YYYY.MM.dd}"
}
}
}


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.