Dear Friends:
I want to use logstash to extract a special part from access.log file.For instance,to extract all the logs that @timestamp=="2017-10-24T16:41:16+08:00" from the target access.log file to a new file.That is to say “File Input to Filter to File Output”.
To make this come true,I installed logstash on the host where the access.log file exists.This logstash application is just as a sieve to select the special part of the access.log and make a new file to store them(the special part),that is to say the Input is access.log file,the Output is New file.
In the first step,I complete the installation of logstash and made a simple test by cmd "bin/logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}' ",this turns out to be all right,it works well.
At the second step,I made a configuration file called logstash_core.conf, this is the core part to accomplish my original idea,by the certain plugins used or configured in this file to make the selection come true.
At the third step,after the configuration,I start the process of logstah by cmd "bin/logstash --verbose -f config/logstash_core.conf --config.reload.automatic &",but the wanted consequence does not appear.
I wonder if this(selection) is feasible.If it is ,where is the mistake in my operation?
The following if the configuration of my logstash application.
#input part
input{
file{
path => "/root/10.200.200.54_access.log" #the path to my access.log file
type => "web_log"
codec => "json"
start_position => "beginning"
}
}
#filter part
filter {
if [type] == "web-log" {
mutate {
gsub => ["url-info", "\x", "\\x"]
}
mutate {
gsub => ["agent", "\x", "\\x"]
}
}
}
filter{
date{
match => [ "@timestamp", "yyyy-MM-dd HH:mm:ss","ISO8601" ]
}
}
#output part
output{
if [@timestamp] == "2017-10-24T16:41:16+08:00" {
file{
path => "/root/target_file.log" ##the path to store my new file.
codec => "json"
}
}
}
Look forward to your response.
Regards.