I want to add a field with filename


(Asier Salueña Sediles) #1

Hi you all, i'm starting a project with elk stack and we have few domains working on the same machine, which sends his apache logs to elk. I want to extract the filename to filter by this field on kibana.
I'm trying something like the following code:

filter {
  if [type] == "apache-access" {
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
      match => { "message" => "%{GREEDYDATA}/%{GREEDYDATA:filename}\.es_access_log"}
    }
    mutate {
      remove_field => [ "message" ]
    }
    }
  }

When I look at Kibana I find out that I have a field called source which contains whole path to the file but It's impossible to me to extract filename from here


(Magnus Bäck) #2

Grok filters stop evaluating expressions when they get a match, so if the COMBINEDAPACHELOG expression matches the other one won't be tried at all. Secondly, if the file path is stored in the source field you obviously need to tell the grok filter to match that field and not message.


(Asier Salueña Sediles) #3

So, I have to apply another grok filter for this document-type, is this correct?


(Magnus Bäck) #4

Yes.


(Asier Salueña Sediles) #5

Thanks for your help, I´m gonna try it.


(Asier Salueña Sediles) #6

I have been doing tests during this week with this configuration and still does not show the filename, I dont know why is it incorrect.
First config file:

filter {
  if [type] == "apache-access" {
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
    }
    mutate {
      remove_field => [ "message" ]
    }
    geoip {
        source => "clientip"
    }
  }
}

Second config file:

filter {
    if [%{host}] == "***" {
        grok {
            match => { "source" => "%{GREEDYDATA}/%{GREEDYDATA:filename}\.es_access_log"}
        }
     }
}

(Magnus Bäck) #7
if [%{host}] == "***" {

[host], not [%{host}]. I assume "***" is your way of censoring the hostname.


(Asier Salueña Sediles) #8

Thanks! Now is working properly.


(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.