Hello,
I have a Ceph storage cluster for S3 Object storage, and the radosgw service uses a Unix socket for logs. I have verified the socket works properly with netcat, and my work around as of late has been to pipe this to a file, then run filebeat. The filebeat solution is pretty clumsy, and using logstash with the logstash.inputs.unix plugin seems much cleaner.
Unfortunately, logstash continues to say "Socket not present" in the logs like so:
[2018-09-19T18:16:54,184][WARN ][logstash.inputs.unix ] Socket not present, wait for seconds for socket to appear {:client=>"/var/run/ceph/opslog"}
[2018-09-19T18:16:59,184][WARN ][logstash.inputs.unix ] Socket not present, wait for seconds for socket to appear {:client=>"/var/run/ceph/opslog"}
[2018-09-19T18:17:04,184][WARN ][logstash.inputs.unix ] Socket not present, wait for seconds for socket to appear {:client=>"/var/run/ceph/opslog"}
I am not sure why this is. A simple: nc -U /var/run/ceph/opslog
presents results
[root@server]# nc -U /var/run/ceph/opslog
{"bucket":"ownersbucket","time":"2018-09-19 17:42:14.110497Z","time_local":"2018-09-19 17:42:14.110497","remote_addr":"10.188.50.107","object_owner":"iamowner","user":"anonymous","operation":"GET","uri":"GET /ownersbucket/test_file.tgz HTTP/1.1","http_status":"200","error_code":"","bytes_sent":8102638,"bytes_received":0,"object_size":8102638,"total_time":126889,"user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.92 Safari/537.36","referrer":""},
maybe I have an incorrect input configuration? Or maybe I am misunderstanding the use of this plugin? Seems pretty straight forward though.
input {
unix {
id => "radosgw"
mode => "client"
path => "/var/run/ceph/opslog"
codec => "json_lines"
}
}
I also verified:
- socket is not in use by someone else
- socket has read permission for everyone
- plugin-input-unix is installed
Any suggestions would be very appreciated! I would love to get away from my clunky netcat -> flat file -> filebeat implementation.
best regards,
jw