'Unix socket not present' in logs, but exists and verified with netcat

Elastic Stack Logstash
1

Hello,

I have a Ceph storage cluster for S3 Object storage, and the radosgw service uses a Unix socket for logs. I have verified the socket works properly with netcat, and my work around as of late has been to pipe this to a file, then run filebeat. The filebeat solution is pretty clumsy, and using logstash with the logstash.inputs.unix plugin seems much cleaner.

Unfortunately, logstash continues to say "Socket not present" in the logs like so:
[2018-09-19T18:16:54,184][WARN ][logstash.inputs.unix ] Socket not present, wait for seconds for socket to appear {:client=>"/var/run/ceph/opslog"}
[2018-09-19T18:16:59,184][WARN ][logstash.inputs.unix ] Socket not present, wait for seconds for socket to appear {:client=>"/var/run/ceph/opslog"}
[2018-09-19T18:17:04,184][WARN ][logstash.inputs.unix ] Socket not present, wait for seconds for socket to appear {:client=>"/var/run/ceph/opslog"}

I am not sure why this is. A simple: nc -U /var/run/ceph/opslog presents results

[root@server]# nc -U /var/run/ceph/opslog
{"bucket":"ownersbucket","time":"2018-09-19 17:42:14.110497Z","time_local":"2018-09-19 17:42:14.110497","remote_addr":"10.188.50.107","object_owner":"iamowner","user":"anonymous","operation":"GET","uri":"GET /ownersbucket/test_file.tgz HTTP/1.1","http_status":"200","error_code":"","bytes_sent":8102638,"bytes_received":0,"object_size":8102638,"total_time":126889,"user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.92 Safari/537.36","referrer":""},

maybe I have an incorrect input configuration? Or maybe I am misunderstanding the use of this plugin? Seems pretty straight forward though.

input {
  unix {
    id => "radosgw"
    mode => "client"
    path => "/var/run/ceph/opslog"
    codec => "json_lines"
  }
}

I also verified:

  • socket is not in use by someone else
  • socket has read permission for everyone
  • plugin-input-unix is installed

Any suggestions would be very appreciated! I would love to get away from my clunky netcat -> flat file -> filebeat implementation.

best regards,
jw

2

The way the plugin gets to this state is if Ruby's File::socket?(path) returns false. Since JRuby uses the Java filesystem bindings, I believe this can occur if the Java Security Manager is enabled and does not provide access to the file at that path. If you create a socket in Logstash's config directory, does the same problem persist?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.