Question about parsing log.file.path field

Elastic Stack Logstash
1

Hi all,

we are trying to ingest from filebeat some differents apache logs from diffent apps and different directories.
So we have this input structure (this are the log.file.path fields ingested)
/home/E879365/logs/app1_azure/app1.log
/home/E879365/logs/app2_was/app2.log
/home/E879365/logs/app3_jboss/app3.log

we need to parse the log.file.path field and get the app qualifier (in this case we nedd app1_azure, app2_was adn app3_jboss)

we have configured this on logstash:

filter {
    grok {
       match => ["log.file.path","/home/E879365/logs/%{DATA:application}/%{GREEDYDATA:resto}"]
        }
}

bu we are not able to get this working....we see docs on elastic but we don't get the application field ingested in any case.

in the docs ingested we see this grokparsefailure on the tags field:
tags on every single doc ingested:
[apache, test, test, beats_input_codec_plain_applied, _grokparsefailure]

we have tried with this, as we have seen a similiar question on the forum, bu it didn't worked either:

filter {
    grok {
       match => ["[log][file][path]","/home/E879365/logs/%{DATA:application}/%{GREEDYDATA:resto}"]
        }
}

On the grok debugger on elastic we see the correct fields parsed:

input: /home/E879365/logs/app1_azure/app1.log
grok pattern: /home/E879365/logs/%{DATA:application}/%{GREEDYDATA:resto}
result:

{
"application": "app1_azure",
"resto": "app1.log"
}

what are we missing or doing wrong?

best regards

Borja

2

Check this and this. Things are much simpler with the dissect plugin.

1 Like
3

Hi Rios!

Thanks for your answer. Links provided gave us the clue to find de solution.

As far as all the input machines were linux we configured logstash in this way:

    mutate {
            copy => {"[log][file][path]" => "filepath" }
           }
    mutate{
         split  => { "filepath" => '/' }
         add_field  => { "application.name" => '%{[filepath][4]}' }
          }

best regards

Borja

1 Like
4

Depend on the case, sometimes is more suitable split, sometimes grok or dissect.

Thank you for your feedback.