Does /usr/share/logstash/data exist?
Yep, this is connecting to the container:
~ $ docker exec -it 38ae222 /bin/bash
bash-4.2$ pwd
/usr/share/logstash
bash-4.2$ ls -l
total 164
drwxr-xr-x. 2 logstash logstash 4096 Apr 12 22:35 bin
drwxr-xr-x. 1 logstash logstash 4096 Jul 23 18:19 config
-rw-r--r--. 1 logstash logstash 2276 Apr 12 22:25 CONTRIBUTORS
drwxr-xr-x. 1 logstash logstash 4096 Jul 23 18:20 data
-rw-r--r--. 1 logstash logstash 3918 Jul 9 19:59 Gemfile
-rw-r--r--. 1 logstash logstash 21598 Jul 9 19:59 Gemfile.lock
drwxr-xr-x. 6 logstash logstash 4096 Apr 12 22:35 lib
-rw-r--r--. 1 logstash logstash 589 Apr 12 22:25 LICENSE
drwxr-xr-x. 4 logstash logstash 4096 Apr 12 22:35 logstash-core
drwxr-xr-x. 3 logstash logstash 4096 Apr 12 22:35 logstash-core-plugin-api
drwxr-xr-x. 4 logstash logstash 4096 Apr 12 22:35 modules
-rw-rw-r--. 1 logstash logstash 28122 Apr 12 22:29 NOTICE.TXT
drwxr-xr-x. 1 logstash logstash 4096 Jul 23 18:19 pipeline
drwxr-xr-x. 3 logstash logstash 4096 Apr 12 22:35 tools
drwxr-xr-x. 1 logstash logstash 4096 Apr 12 22:35 vendor
bash-4.2$ ls -l data/
total 32
drwxr-xr-x. 2 logstash logstash 4096 Jul 23 18:20 dead_letter_queue
drwxr-xr-x. 2 logstash logstash 4096 Jul 23 18:20 queue
-rw-r--r--. 1 logstash logstash 36 Jul 23 18:20 uuid
-rw-r--r--. 1 logstash logstash 172 Jul 19 01:31 zmx.conf
bash-4.2$
That . at the end of the permissions may indicate an alternate access method such as an access control list applies to the file. The details will depend on your OS.
ok.
I am using the logstash container from the elastic.co repository:
docker.elastic.co/logstash/logstash:6.2.4
If I look at the permissions of all the files in folder, including the ones that were created today it would appear they have the same permissions:
bash-4.2$ getfacl --all-effective data/*
# file: data/dead_letter_queue
# owner: logstash
# group: logstash
user::rwx
group::r-x
other::r-x
# file: data/queue
# owner: logstash
# group: logstash
user::rwx
group::r-x
other::r-x
# file: data/uuid
# owner: logstash
# group: logstash
user::rw-
group::r--
other::r--
# file: data/zmx.conf
# owner: logstash
# group: logstash
user::rw-
group::r--
other::r--
bash-4.2$
If I examine the permissions of the /usr/share/logstash/config/ files, which we know logstash is able to read I see the permissions are the same:
bash-4.2$ getfacl --all-effective data/*
# file: data/dead_letter_queue
# owner: logstash
# group: logstash
user::rwx
group::r-x
other::r-x
# file: data/queue
# owner: logstash
# group: logstash
user::rwx
group::r-x
other::r-x
# file: data/uuid
# owner: logstash
# group: logstash
user::rw-
group::r--
other::r--
# file: data/zmx.conf
# owner: logstash
# group: logstash
user::rw-
group::r--
other::r--
bash-4.2$
I didn't modify the user's permissions in the container, you can see from the Dockerfile.
Just to check for the location of the file, I moved it back to the '/usr/share/logstash/' directory again to test and I'm still seeing the same error message.
[2018-07-23T19:45:45,308][ERROR][logstash.inputs.jmx ] No such file or directory - No such directory: /usr/share/logstash/zmx.conf
I understand that JMX may not be a popular plugin to use. Are there any additional restrictions I should be aware of? Does it not work in a docker image? I can try installing it on a virtual server and see if I get the same results.
I do sincerely appreciate all of your efforts to assist Badger! Definitely could not have made it this far without you.
I have discovered that the only problem I had with the file not found error is my inability to read and follow the excellent documentation provided (Jmx input plugin | Logstash Reference [8.11] | Elastic).
...a directory absolute path containing json files...
I assumed it was looking for a filename not a directory. Obviously I was wrong.
After I created a jmx specific folder and dropped my config files there (as documented) I am no longer getting an error regarding the configuration of logstash but it's not emiting any events.
I'm only seeing this warning when trying to parse the jmx configuration file:
[2018-07-24T01:54:26,415][WARN ][logstash.inputs.jmx ] Issue loading configuration from file {:file=>"/usr/share/logstash/jmx/server", :exception=>"Unexpected character ('"' (code 34)): was expecting comma to separate Object entries\n at [Source: (byte)"{\n "host" : "10.10.10.10",\n "port" : 7051,\n "username" : "user",\n "password": "pass",\n "alias" : "server.my.domain"\n "queries" : [\n {\n "object_name" : "Catalina:type=Manager,host=localhost,context=/mywebapp1",\n "attributes" : [ "activeSessions" ],\n }, {\n "object_name" : "Catalina:type=Manager,host=localhost,context=/mywebapp2",\n "attributes" : [ "activeSessions" ],\n }, {\n "object_name" : "Catalina:type=Manager,host=localhost,context=/mywebapp3",\n"[truncated 308 bytes]; line: 7, column: 4]", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/json.rb:17:in
jruby_load'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-jmx-3.0.6/lib/logstash/inputs/jmx.rb:332:in
block in run'", "org/jruby/RubyDir.java:383:inforeach'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-jmx-3.0.6/lib/logstash/inputs/jmx.rb:326:in
run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:514:ininputworker'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:507:in
block in start_input'"]}
Any chance someone may understand what is missing?
Maybe a second pair of eyes can confirm where the file has the incorrect syntax:
{
"host" : "10.10.10.10",
"port" : 7051,
"username" : "user",
"password": "pass",
"alias" : "server.my.domain"
"queries" : [
{
"object_name" : "Catalina:type=Manager,host=localhost,context=/mywebapp1",
"attributes" : [ "activeSessions" ],
}, {
"object_name" : "Catalina:type=Manager,host=localhost,context=/mywebapp2",
"attributes" : [ "activeSessions" ],
}, {
"object_name" : "Catalina:type=Manager,host=localhost,context=/mywebapp3",
"attributes" : [ "activeSessions" ],
}, {
"object_name" : "Catalina:type=Manager,host=localhost,context=/mywebapp4",
"attributes" : [ "activeSessions" ],
}, {
"object_name" : "Catalina:type=Manager,host=localhost,context=/mywebapp5"
"attributes" : [ "activeSessions" ]
} ]
}
There should not be a comma at the end of that line (or the 3 others like it).
For all of the entries identifying the host, port, username, password, alias. They must also have a comma after them. All of the entries for object_name must also have a comma when you have attributes following them.
Thanks a lot for all your help on this one!
I need to read more on the syntax utilized within the configuration files. Probably wouldn't have had as much trouble with this as I have.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.