Hi Team,
I am trying to parse jenkins build logs using filebeat. There are several subdirectories that it needs to traverse to. I can also see in the filebeat log what all locations filebeat is checking. However when i create a new log file or add entry to existing log files, logstash is not harvesting those changes. Please help. If i use just a single path instead of 3 file path in my case, it seems to work.
My Filebeat config
[ashivaramakrishnan@ci001.atl1.turn.com filebeat]$ more conf.d/jenkins-ci.yml
filebeat:
inputs:
- paths:
- "/var/lib/jenkins/jobs/cd-pipeline-master*/builds/*/junitResult.xml"
- "/var/lib/jenkins/jobs/cd-pipeline-release*/builds/*/junitResult.xml"
- "/var/lib/jenkins/jobs/cd-pipeline-production*/builds/**/junitResult.xml"
encoding: plain
fields:
timezone: PDT
log_type: jenkins_junit
close_renamed: true
close_removed: true
fields_under_root: true
type: log
ignore_older: 5m
scan_frequency: 10s
harvester_buffer_size: 16384
Filebeat log files::
Configured paths: [
/var/lib/jenkins/jobs/cd-pipeline-master*/builds/*/junitResult.xml
/var/lib/jenkins/jobs/cd-pipeline-release*/builds/*/junitResult.xml
/var/lib/jenkins/jobs/cd-pipeline-production*/builds/junitResult.xml
/var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/*/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/*/*/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/*/*/*/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/*/*/*/*/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/*/*/*/*/*/*/junitResult.xml]
For example if try modifying a file in the below path. filebeat doesnt harvest the file.
eg::
/var/lib/jenkins/jobs/cd-pipeline-master-ucheetahclient-test/builds/0000/junitResult.xml
Thanks
Arvind