Context
I am building a program that will pass an Apache log file into Logstash and output the result (after parsing and filtering) to an external database (Elastic, MongoDB, etc.).
Basically, the program will execute following command:
gunzip -c -k "somefile.gz" | logstash -f "logstash.conf"
With logstash.conf containing:
input {
stdin {}
}
filter {
# ...
}
output {
mongodb {
# ...
}
}
But, to help future debug/forensic, I want to grab whatever Logstash says during it's process.
By default, it seems Logstash outputs it's internal log messages to stdout, as an example, the following command:
logstash -e "input { generator { count => 3 } } output { null {} }"
Outputs:
Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2018-03-28T22:42:53,484][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[2018-03-28T22:42:53,515][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[2018-03-28T22:42:54,222][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-03-28T22:42:55,092][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.2.3"}
[2018-03-28T22:42:55,795][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2018-03-28T22:42:58,225][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-03-28T22:42:58,407][INFO ][logstash.pipeline ] Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x35d85de2 run>"}
[2018-03-28T22:42:58,559][INFO ][logstash.agent ] Pipelines running {:count=>1, :pipelines=>["main"]}
[2018-03-28T22:42:58,867][INFO ][logstash.pipeline ] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#<Thread:0x35d85de2 run>"}
Adding --log.level=debug gives at lot more: the same as previous command, plus a lot more, including the following message-related log:
[2018-03-28T22:44:30,671][DEBUG][logstash.pipeline ] filter received {"event"=>{"@timestamp"=>2018-03-28T22:44:30.567Z, "sequence"=>1, "@version"=>"1", "message"=>"Hello world!", "host"=>"5e26a941934f"}}
[2018-03-28T22:44:30,679][DEBUG][logstash.pipeline ] output received {"event"=>{"@timestamp"=>2018-03-28T22:44:30.567Z, "sequence"=>1, "@version"=>"1", "message"=>"Hello world!", "host"=>"5e26a941934f"}}
[2018-03-28T22:44:30,680][DEBUG][logstash.pipeline ] filter received {"event"=>{"@timestamp"=>2018-03-28T22:44:30.568Z, "sequence"=>2, "@version"=>"1", "message"=>"Hello world!", "host"=>"5e26a941934f"}}
...
Lots of noise but it will be useful when tracking down an error.
Problem
I had trouble directly capturing the stdout and stderr of the logstash command in my program, but anyway, I find it cleaner to ask Logstash to directly write to a log file (that I would then read in my program).
According to the documentation adding the --path.logs=/tmp to the logstash command should make it write log file to /tmp but it does not. I get no log file in /tmp after the following command ends (nor during it executes):
logstash \
--path.logs=/tmp \
--log.level=debug \
-e "input { generator { count => 3 } } output { null {} }"
I've also tried the conffiguration file way without much success:
- added
log.level: debugandpath.logs: /tmp/logstash-logsinto the/etc/logstash/logstash.ymlfile. - created the /tmp/logstash-logs directory (with 777 permissions)
- and executed
logstash -e "input { generator { count => 3 } } output { null {} }"
The logstash outputs confirms it took the changes into account (Sending Logstash's logs to /tmp/logstash-logs which is now configured via log4j2.properties) but still no log files in /tmp/logstash-logs.
So: How make logstash write it's log messages (not the data it processes) to a file?