Condition on filename in the path

[Hyrastios]

Hi.

I want to parse all my apache_logs in the same time, but to do so I have to differentiate logs coming from access files, error files, or mod_jk files.

I'm using filebeat to import my logs and I send them to elasticsearch with the following pipeline.conf file :

input {
    beats {
        port => "5044"
	}
}
filter {
	grok{
	    patterns_dir => ["C:\logstash-7.0.1\patterns\extra"]
		match => {"message" => "%{GREEDYDATA:test}"}
	}
	if [path] =~ "access"{
		mutate {
			add_field => {"test_field" => "I'm just here for a test"}
		}
	}
}
output {
	elasticsearch {
		hosts => ["localhost:9200", "localhost:9201"]
		index => "tests_apache_cond"
	}
	stdout { codec => rubydebug }
}

But it doesn't add any field.

Moreover when I try to use the condition "if ![path]", the condition is fullfilled just as if the path field didn't exist (same with log.file.path).

After having tried everything that came to my mind I'm asking you guys if you have an idea for me.

Thanks a lot for taking the time to answer my question (and sorry for my approximative english :s)

[logger (Malte Leek)]

It sound like there is no path field, maybe it is a "source" field. they changed it with some release.

could you show us a original input log.

[Hyrastios]

Thanks for your quick reply !

I just found a solution on my own but thanks for your help !

By reading the Filebeat documentation once again I noticed that you can define more than one input and apply a specific tag on each. That's what my filebeat.yml > filebeat.inputs looks like now

- type: log
  enabled: true
  paths:
    - C:\Users\ET02910\ElasticSearch\apache_tests\access*
  tags: ["log access"]
  
- type: log
  enabled: true
  paths:
    - C:\Users\ET02910\ElasticSearch\apache_tests\error*
  tags: ["log error"]
  
- type: log
  enabled: true
  paths:
    - C:\Users\ET02910\ElasticSearch\apache_tests\mod_jk*
  tags: ["log mod_jk"]

And then I just have to check my condition on the tags field which is accessible from Logstash :

if "log access" in [tags]{
		mutate {
			add_field => {"test_field" => "I'm just here for a test"}
		}
	}

Concerning the source field you were talking about, I tried with source and _source before posting here but none of them worked.

Thanks for your help anyway,
Have a nice week-end !