I have a cluster of 3 elastic search nodes + kibana vm - version 6.2.4 running on 4 vms of Centos 7 . Logstash instances running on every elasticsearch nodes
The logstash service is running, but any indices have been created and pipelines are not created
Here my logstash.yml:
node.name: l-elk20
path.data: /var/lib/logstash
pipeline.id: main
pipeline.workers: 2
pipeline.batch.size: 125
pipeline.batch.delay: 50
pipeline.unsafe_shutdown: false
config.test_and_exit: false
config.reload.automatic: true
config.reload.interval: 3s
config.debug: true
config.support_escapes: false
queue.type: memory
queue.page_capacity: 64mb
queue.max_events: 0
queue.max_bytes: 1024mb
queue.checkpoint.acks: 1024
queue.checkpoint.writes: 1024
queue.checkpoint.interval: 1000
http.host: "l-elk-20.lab.local"
http.port: 9600-9700
log.level: debug
path.logs: /var/log/logstash
# Enable X-Pack Monitoring
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.url: [ "https://l-elk20.lab.local:9200","https://l-elk21.lab.local:9200","https://l-elk22.lab.local:9200" ]
xpack.monitoring.elasticsearch.ssl.ca: /etc/logstash/config/certs/ca.crt
xpack.management.enabled: true
xpack.management.elasticsearch.url: [ "https://l-elk20.lab.local:9200","https://l-elk20.lab.local:9200","https://l-elk20.lab.local:9200" ]
xpack.management.elasticsearch.ssl.ca: /etc/logstash/config/certs/ca.crt
xpack.management.pipeline.id: [ "main" ]
xpack.monitoring.elasticsearch.username: elastic
xpack.monitoring.elasticsearch.password: password
xpack.management.elasticsearch.username: elastic
xpack.management.elasticsearch.password: password
Here is conf.d/example.conf:
input {
beats {
port => 11000
ssl => true
ssl_key => '/etc/logstash/config/certs/l-elk20.lab.local.pkcs8.key'
ssl_certificate => '/etc/logstash/config/certs/l-elk20.lab.local.crt'
}
}
output {
elasticsearch {
hosts => ["https://l-elk20.lab.local:9200","https://l-elk21.lab.local:9200","https://l-elk22.lab.local:9200"]
cacert => '/etc/logstash/config/certs/ca.crt'
user => 'elastic'
password => 'password'
index => 'lab-linux-syslog-%{+YYYY.MM.dd}'
}
}
And here is pipelines.yml :
# This file is where you define your pipelines. You can define multiple.
# For more information on multiple pipelines, see the documentation:
# https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html
- pipeline.id: main
path.config: "/etc/logstash/conf.d/*.conf"
When I starting logstash with systemctl start logstash
getting error in /var/log/logstash/logstash-plain.log :
[2018-06-18T11:55:13,799][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<SocketError: initialize: name or service not known>, :backtrace=>["org/jruby/ext/socket/RubyTCPServer.java:124:ininitialize'", "org/jruby/RubyIO.java:875:in new'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/puma-2.16.0-java/lib/puma/binder.rb:234:inadd_tcp_listener'", "(eval):2:in add_tcp_listener'", "/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:88:instart_webserver'", "/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:44:in block in run'", "org/jruby/RubyRange.java:485:ineach'", "org/jruby/RubyEnumerable.java:1067:in each_with_index'", "/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:39:inrun'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:374:in block in start_webserver'"]} [2018-06-18T11:55:13,811][WARN ][logstash.configmanagement.elasticsearchsource] Restored connection to ES instance {:url=>"https://elastic:xxxxxx@l-elk20.lab.local:9200/"} [2018-06-18T11:55:13,815][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (SystemExit) exit
Could somebody take a look on this and suggest me what I can do here?
Thanks