Hey all, I'm a new logstash user. I'm trying to ease in a client to ELK who is currently analyzing their logs using an existing tool. I wanted to start by just using logstash for log shipping and aggregating them to one server, and then add on the EK portion to show their value later without disrupting their current process.
(Difficulty: Windows)
So I set up the server and logstash-forwarder and have both of them running, and extracting just the messages from the logs on the server end. But what I can't figure out how to do is to split them back out into their original filenames instead of one jumbo file with all the messages in it.
Desired goal - from a bunch of servers (hostname1...hostnameN), pull a bunch of log files and put them on the logstash server with date rotation. So hostname1:C:\file1.log becomes server:C:\logs\hostname1\file1.log.2015-07-10 et al.
Here's my logstash-forwarder.conf
{
"network": {
"servers": [
"localhost:12345"
],
"ssl certificate": "C:/logstash-1.5.1/logstash.crt",
"ssl key": "C:/logstash-1.5.1/logstash.key",
"ssl ca": "C:/logstash-1.5.1/logstash.crt",
"timeout": 15
},
"files": [
{
"paths": [
"C:/Program Files/uptime software/uptime/logs/*"
]
}
]
}
And my logstash.conf
input {
lumberjack {
# The port to listen on
port => 12345
ssl_certificate => "C:/logstash-1.5.1/logstash.crt"
ssl_key => "C:/logstash-1.5.1/logstash.key"
# Set this to whatever you want.
type => "remotelogs"
}
}
output {
file {
path => "C:/logstash-1.5.1/logs/%{host}/log.%{file}.%{+yyyy.MM.dd.HH}"
message_format => "%{message}"
}
}
This gives me a mkdir error when I try to start the logstash server. I've fooled around with different options for the path and it'll put all the lines in one file fine, I just want to end up with the original filenames (though ideally with date rotation).
Thanks!
Ernest