Hello,
I am seeking some guidance regarding the configuration of the logstash jmx input plugin. I apologize for the formatting/wording of this as I'm quite certain it's wrong. Hopefully you can overlook this and provide assistance in spite of my ignorance.
I have a single logstash docker container running logstash 6.2.4
The Dockerfile is formatted similar to:
FROM docker.elastic.co/logstash/logstash:6.2.4
RUN /usr/share/logstash/bin/logstash-plugin install logstash-input-jmx
RUN rm -f /usr/share/logstash/pipeline/logstash.conf
RUN rm -f /usr/share/logstash/pipeline/pipelines.yml
RUN rm -f /usr/share/logstash/config/logstash.yml
COPY --chown=logstash:logstash logstash.conf /usr/share/logstash/pipeline/
COPY --chown=logstash:logstash pipelines.yml /usr/share/logstash/pipeline/
COPY --chown=logstash:logstash config1 config2 config3 /usr/share/logstash/pipeline/
COPY --chown=logstash:logstash logstash.yml /usr/share/logstash/config/
I chose to follow the example configuration provided on: Jmx input plugin | Logstash Reference [8.11] | Elastic
My logstash.conf:
input {
jmx {
path => "/usr/share/logstash/pipeline/lmsgrad"
polling_frequency => 600
type => "jmx"
nb_thread => 4
}
}output {
elasticsearch {
hosts => ["http://elasticsearch:9200"]
index => "jmx-logs-%{+YYYY.MM.dd}"
}
}
My pipelines.yml:
- pipeline.id: main
path.config: "/usr/share/logstash/pipeline"
path.config: "/usr/share/logstash/pipeline/config1"
path.config: "/usr/share/logstash/pipeline/config2"
path.config: "/usr/share/logstash/pipeline/config3"
My config1-3 is similar to:
{
//Required, JMX listening host/ip
"host" : "10.10.10.10",
//Required, JMX listening port
"port" : 9012,
//Optional, the username to connect to JMX
"username" : "user",
//Optional, the password to connect to JMX
"password": "pass",
//Optional, use this alias as a prefix in the metric name. If not set use _
"alias" : "test.homeserver.elasticsearch",
//Required, list of JMX metrics to retrieve
"queries" : [
{
//Required, the object name of Mbean to request
"object_name" : "java.lang:type=Memory",
//Optional, use this alias in the metrics value instead of the object_name
"object_alias" : "Memory"
}, {
"object_name" : "java.lang:type=Runtime",
//Optional, set of attributes to retrieve. If not set retrieve
//all metrics available on the configured object_name.
"attributes" : [ "Uptime", "StartTime" ],
"object_alias" : "Runtime"
}, {
//object_name can be configured with * to retrieve all matching Mbeans
"object_name" : "java.lang:type=GarbageCollector,name=",
"attributes" : [ "CollectionCount", "CollectionTime" ],
//object_alias can be based on specific value from the object_name thanks to ${}.
//In this case ${type} will be replaced by GarbageCollector...
"object_alias" : "${type}.${name}"
}, {
"object_name" : "java.nio:type=BufferPool,name=",
"object_alias" : "${type}.${name}"
} ]
}
My logstash.yml is similar to:
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.url: http://elasticsearch:9200
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: changeme
Files have been modified to protect the hosts & passwords as required by common sense.
Running this container with the above configuration provides me with the following error:
[2018-07-16T22:22:54,430][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:167:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:305:in `block in converge_state'"]}
This unfortunately does not provide a lot of guidance regarding where the error is.
Would anyone happen to have any suggestions?
Thanks.