Centralized pipeline management - pipeline terminating

Hello community,

I am trying to use the centralize pipeline management feature , I followed the document https://www.elastic.co/guide/en/logstash/6.0/logstash-centralized-pipeline-management.html
I have a x-pack licence aswell. but after creating the pipeline using kibana UI -> logstash ->pipeline when I start the logstash I am getting this

Pipeline terminated {"pipeline.id"=>".monitoring-logstash"} logstash_test exited with code 0

`    logstash.config.source.local.configpathloader] No config files found in path {:path=>"/usr/share/logstash/pipeline/*"}`

I know this error is coming because of no pipeline configuration files in the logstash pipeline directory but I guess that is not the case if I am using centralized pipeline management

and

if I use the same pipeline configuration in logstash/pipeline directory it is working fine. logstash not giving any errors. what might be the cause?

here is my elasticsearch.yml file

**node.name: elastic-test **
cluster.name: "elk-cluster"
network.host: 0.0.0.0
network.bind_host: 0.0.0.0
transport.tcp.port: 9300
http.port: 9200
network.publish_host: ...
discovery.zen.minimum_master_nodes: 1
discovery.type: single-node
xpack.security.enabled: false

Kibana.yml

server.host: "0"
server.name: kibana-test
elasticsearch.url: "http://elasticsearch_test:9200"
elasticsearch.username: ######
elasticsearch.password:######
xpack.monitoring.ui.container.elasticsearch.enabled: true
#xpack.security.enabled: true

Logstash.yml

node.name: logstash-1
http.host: "0.0.0.0"
path.config: /usr/share/logstash/pipeline
config.reload.automatic: true
#xpack.monitoring.enabled: true
#xpack.monitoring.elasticsearch.url: "http://#.#.#.#:9200"
#xpack.monitoring.elasticsearch.username: logstash_system
#xpack.monitoring.elasticsearch.password: changeme
xpack.management.enabled: true
xpack.management.elasticsearch.url: "http://#.#.#.#:9200/"
xpack.management.elasticsearch.username: logstash_admin_user
xpack.management.elasticsearch.password: elastic
xpack.management.logstash.poll_interval: 5s
xpack.management.pipeline.id: ["paloalto", "winlogbeats"]

My docker-compose file

`version: '2.2'

services:

######## ELASTICSEARCH ######################

elasticsearch:
build: /usr/share/elk/elasticsearch/
container_name: elasticsearch_test
environment:
- cluster.name=elk-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- ELASTIC_PASSWORD=changeme
ulimits:
nofile:
soft: 65536
hard: 65536
memlock:
soft: -1
hard: -1
mem_limit: 1g
ports:
- 9300:9300
- 9200:9200
restart: always
volumes:
- es_data1:/usr/share/elasticsearch/data
- /usr/share/elk/configs/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
networks:
- elk

########### LOGSTASH ###########

logstash:
build: /usr/share/elk/logstash/
container_name: logstash_test
environment:
- cluster.name=elk-cluster
- bootstrap.memory_lock=true
- "LS_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
nofile:
soft: 65536
hard: 65536
memlock:
soft: -1
hard: -1
mem_limit: 1g
ports:
- 5000:5000
- 1514:1514
- 5044:5044
- 7575:7575
volumes:
- ls_data1:/usr/share/logstash/data
- /usr/share/elk/configs/logstash.yml:/usr/share/config/logstash.yml
- /usr/share/elk/logstash/pipeline:/usr/share/logstash/pipeline
networks:
- elk
depends_on:
- elasticsearch

################ KIBANA ###############

kibana:
build: /usr/share/elk/kibana/
container_name: kibana_test
environment:
- cluster.name=elk-cluster
#- "ELASTICSEARCH_URL=http://elasticsearch:9200"
mem_limit: 1g
volumes:
- kibana_data1:/usr/share/kibana/data
- /usr/share/elk/configs/kibana.yml:/usr/share/kibana/config/kibana.yml
ports:
- "5601:5601"
networks:
- elk
depends_on:
- elasticsearch

volumes:
es_data1:
driver: local
ls_data1:
driver: local
kibana_data1:
driver: local
networks:
elk:
driver: bridge`

When you are using centralized configuration management, can you try commenting out the path.config line in your logstash.yml file?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.