Hi all,
I'm testing an Azure Cloud deployement (Western Europe) and try to use the Kibana centralized Pipelines feature.
I have a local Logstash instance that I want to register on my Cloud deployment.
Local Logstash instance version is 6.8.3 running on centos.
The local logstash.yml is set as follows
path.data: /var/lib/logstash
cloud.id: deploy_01:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
cloud.auth: elastic:xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
path.logs: /var/log/logstash
xpack.management.enabled: true
xpack.management.pipeline.id: ["pipeline_01"]
xpack.management.elasticsearch.username: elastic
xpack.management.elasticsearch.password: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xpack.management.elasticsearch.hosts: ["https://xxxxxxxxxxxxxxxxxxxxxxx.westeurope.azure.elastic-cloud.com:9243/"]
xpack.management.logstash.poll_interval: 5s
I have created a pipeline on the Kibana Pipelines UI with ID = "pipeline_01" and a basic configuration
input {
beats {
port => 5044
}
}
filter {
}
output {
stdout { codec => rubydebug }
}
When I run Logstash I keep on having the following message
[2019-10-07T09:27:51,527][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
Full log is
[2019-10-07T09:27:33,753][INFO ][logstash.configmanagement.bootstrapcheck] Using Elasticsearch as config store {:pipeline_id=>["pipeline_01"], :poll_interval=>"5000000000ns"}
[2019-10-07T09:27:35,229][INFO ][logstash.configmanagement.elasticsearchsource] Configuration Management License OK
[2019-10-07T09:27:35,474][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.8.3"}
[2019-10-07T09:27:36,034][INFO ][logstash.configmanagement.elasticsearchsource] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elastic:xxxxxx@xxxxxxxxxxxxxxxxxxxxxxxxxxxx.westeurope.azure.elastic-cloud.com:9243/]}}
[2019-10-07T09:27:36,133][WARN ][logstash.configmanagement.elasticsearchsource] Restored connection to ES instance {:url=>"https://elastic:xxxxxx@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx.westeurope.azure.elastic-cloud.com:9243/"}
[2019-10-07T09:27:36,240][INFO ][logstash.configmanagement.elasticsearchsource] ES Output version determined {:es_version=>7}
[2019-10-07T09:27:36,240][WARN ][logstash.configmanagement.elasticsearchsource] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2019-10-07T09:27:36,441][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2019-10-07T09:27:36,632][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2019-10-07T09:27:41,535][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2019-10-07T09:27:46,529][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2019-10-07T09:27:51,527][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
I'm not sure where the issue is... in the local logstash.yml ? or I'm just not properly using the centralized Pipeline all together ?
Any help would be appreciated !
Many thanks