Extract File Name from source

Elastic Stack Logstash
1

Hello,

I am trying to read log file using Filebeat from an application server and write to a remote directory using Logstash.

I am not able to extract the filename from the source field. Need help to extract the file name so use the same file name as output.

stdout:

Sending logstash logs to /var/log/logstash/logstash.log.
{
"message" => "2017-01-30 16:21:32,276 INFO accessLogger - Tracking_Id:1, User_Name:apple, Service_Name:WCustomerService, Method_Name:wEGetCustomerSeasonalStatus, Authenticated:true",
"@version" => "1",
"@timestamp" => "2017-01-30T21:21:39.750Z",
"source" => "/app/tomcat8080/logs/DATAWAR_access.2017-01-30.log",
"offset" => 5761,
"type" => "log",
"input_type" => "log",
"beat" => {
"hostname" => "daspoc-wc-a01d.sys.abc.net",
"version" => "5.1.2",
"name" => "daspoc-wc-a01d.sys.abc.net"
},
"host" => "daspoc-wc-a01d.sys.abc.net",
"tags" => [
[0] "beats_input_codec_plain_applied",
[1] "_grokparsefailure"
]
}

logstash.conf
input {
beats {
port => 5000
}
}

filter {
grok {
match => ["path","/app/tomcat8080/logs/%{DATA:filename}.log"]
}
}
output {
file {
path =>"/var/log/logstash/%{filename}"
codec => line { format => "%{message}"}
}
stdout { codec => rubydebug }
}

2

Please ignore...found the soln.

input {
beats {
port => 5000
}
}

filter {
ruby {
code => "event['filename'] = event['source'].split('/').last"
}
}

output {

  file {
           path =>"/var/log/logstash/%{filename}"
           codec => line { format => "%{message}"}
   }

    stdout { codec => rubydebug }

}

3

Your original grok filter looked fine except that it tried to match against the non-existent path field instead of the source field.

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.