Hello everyone,
I have a cluster of four nodes running - secured with certificates.
I have followed the tutorials to include a logstash to the cluster. At the moment I don't even use /conf.d/* configuration files to allow me to start it more easily ...
I have already tried to use superuser for communication and I have even re-generated all the different certificates for each node but it still says exactly the same thing.
[2021-02-17T22: 26: 17,191] [INFO] [logstash.monitoring.internalpipelinesource] Monitoring License OK
[2021-02-17T22: 26: 17,199] [INFO] [logstash.monitoring.internalpipelinesource] Validated license for monitoring. Enabling monitoring pipeline.
[2021-02-17T22: 26: 17,521] [INFO] [logstash.config.source.local.configpathloader] No config files found in path {: path => "/ etc / logstash / conf.d / *. Conf" }
[2021-02-17T22: 26: 19,530] [INFO] [org.reflections.Reflections] Reflections took 46 ms to scan 1 urls, producing 23 keys and 47 values
[2021-02-17T22: 26: 19,784] [WARN] [logstash.outputs.elasticsearchmonitoring] [. Monitoring-logstash] ** WARNING ** Detected UNSAFE options in elasticsearch output configuration!
** WARNING ** You have enabled encryption but DISABLED certificate verification.
** WARNING ** To make sure your data is secure change: ssl_certificate_verification to true
[2021-02-17T22: 26: 19,852] [INFO] [logstash.outputs.elasticsearchmonitoring] [. Monitoring-logstash] Elasticsearch pool URLs updated {: changes => {: removed => [],: added => [https : // grillotron: xxxxxx@prso-elk-viten-1.elk.prso.net: 9200 /]}}
[2021-02-17T22: 26: 19,945] [WARN] [logstash.outputs.elasticsearchmonitoring] [. Monitoring-logstash] Restored connection to ES instance {: url => "https: // grillotron: xxxxxx @ prso-elk- viten-1.elk.prso.net:9200/ "}
[2021-02-17T22: 26: 19,964] [INFO] [logstash.outputs.elasticsearchmonitoring] [. Monitoring-logstash] ES Output version determined {: es_version => 7}
[2021-02-17T22: 26: 19,974] [WARN] [logstash.outputs.elasticsearchmonitoring] [. Monitoring-logstash] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {: es_version => 7}
[2021-02-17T22: 26: 20,116] [INFO] [logstash.outputs.elasticsearchmonitoring] [. Monitoring-logstash] New Elasticsearch output {: class => "LogStash :: Outputs :: ElasticSearchMonitoring",: hosts => [ "https://prso-elk-viten-1.elk.prso.net:9200"]}
[2021-02-17T22: 26: 20,135] [WARN] [logstash.javapipeline] [. Monitoring-logstash] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
[2021-02-17T22: 26: 20,250] [INFO] [logstash.javapipeline] [. Monitoring-logstash] Starting pipeline {: pipeline_id => ". Monitoring-logstash", "pipeline.workers" => 1, "pipeline .batch.size "=> 2," pipeline.batch.delay "=> 50," pipeline.max_inflight "=> 2," pipeline.sources "=> [" monitoring pipeline "],: thread =>" # < Thread: 0x7924cb75 run> "}
[2021-02-17T22: 26: 21,070] [INFO] [logstash.javapipeline] [. Monitoring-logstash] Pipeline Java execution initialization time {"seconds" => 0.81}
[2021-02-17T22: 26: 21,137] [INFO] [logstash.javapipeline] [. Monitoring-logstash] Pipeline started {"pipeline.id" => ". Monitoring-logstash"}
[2021-02-17T22: 26: 21,212] [INFO] [logstash.agent] Pipelines running {: count => 1,: running_pipelines => [: ". Monitoring-logstash"],: non_running_pipelines => []}
[2021-02-17T22: 26: 21,498] [INFO] [logstash.agent] Successfully started Logstash API endpoint {: port => 9600}
[2021-02-17T22: 26: 23,127] [INFO] [logstash.javapipeline] [. Monitoring-logstash] Pipeline terminated {"pipeline.id" => ". Monitoring-logstash"}
[2021-02-17T22: 26: 23,478] [INFO] [logstash.runner] Logstash shut down.
I don't know what to try anymore
You don't see to have any pipeline configured, if you do not have any pipeline running logstash will shutdown.
Can you share your logstash.yml, pipelines.yml and how are you starting logstash? Are you running it as a service or using the command line?
Sure - here is logstash.yml
node.name: logstash1.elk.prso.net
path.data: /var/lib/logstash
queue.max_bytes: 8gb
path.logs: /var/log/logstash
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: '1234567890'
xpack.monitoring.elasticsearch.hosts: [ 'https://prso-elk-viten-1.elk.prso.net:9200' ]
xpack.monitoring.elasticsearch.ssl.certificate_authority: /etc/logstash/certs/ca.crt
And here pipelines.yml
- pipeline.id: main
path.config: "/etc/logstash/conf.d/*.conf"
In the directory /etc/logstash/conf.d/
root@prso-elk-viten-1:~# ls -lsa /etc/logstash/conf.d/
total 16
4 drwxrwxr-x 4 root root 4096 Feb 17 17:56 .
4 drwxrwxr-x 5 root root 4096 Feb 18 09:52 ..
4 drwxr-xr-x 2 root root 4096 Feb 14 23:23 patterns
4 drwxr-xr-x 2 root root 4096 Feb 17 17:56 temp
In temp i have some cofigutarion files
registro.conf - unactivated by dont stay direcly at "config.d" directory
In Patterns - GROK patterns
--------------------------------------------------------------
Logstash is running like a service (on debian)
Thanks
So your pipelines need to be inside /etc/logstash/conf.d/ and have the *.conf extension.
But in your /etc/logstash/conf.d/ you do not have any .conf file, so you do not have any pipeline.
Without a pipeline logstash has nothing to do and will shutdown, which is waht you are seeing in your logs, you need to create a pipeline to use logstash.
Ok solved
I also had some errors in my only configuration file so I did NOT catch it and therefore, as it did not have work, it turned off.
Thanks ;)
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.