Logstash cannot identify config file, it stops after starting , i am using docker desktop

so , i wrote an elasticsearch.yaml , which contains the configuration of elasticsearch, kibana and logstash. i will just attach the text

version: '3.3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.4.0
    container_name: elasticsearch
    restart: always
    environment:
      - xpack.security.enabled=false
      - discovery.type=single-node
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    cap_add: 
      - IPC_LOCK
    volumes:
      - elasticsearch-data:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
  kibana:
    container_name: kibana
    image: docker.elastic.co/kibana/kibana:7.4.0
    restart: always
    environment: 
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
    ports:
      - 5601:5601
    depends_on:
      - elasticsearch
  Logstash:
    image: logstash:8.7.0
    container_name: logstash
    restart: always
    # volume: 
    #   - ./logstash/:/logstash_dir
    command: logstash -f C:\Users\svats\Downloads\logstash-8.7.0\logstash.conf
    depends_on:
      - elasticsearch
    environment:
      LS_JAVA_OPTS: "-Xmx256m -Xms256m"
volumes:
  elasticsearch-data:

// then the logs in logstash are as follows:


Log4j configuration path used is: /usr/share/logstash/config/log4j2.properties
2023-05-17 03:44:05 [2023-05-16T22:14:05,360][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.7.0", "jruby.version"=>"jruby 9.3.10.0 (2.6.8) 2023-02-01 107b2e6697 OpenJDK 64-Bit Server VM 17.0.6+10 on 17.0.6+10 +indy +jit [x86_64-linux]"}
2023-05-17 03:44:05 [2023-05-16T22:14:05,362][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Dls.cgroup.cpuacct.path.override=/, -Dls.cgroup.cpu.path.override=/, -Xmx256m, -Xms256m, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
2023-05-17 03:44:05 [2023-05-16T22:14:05,524][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
2023-05-17 03:44:05 [2023-05-16T22:14:05,916][WARN ][deprecation.logstash.monitoringextension.pipelineregisterhook] Internal collectors option for Logstash monitoring is deprecated and targeted for removal in the next major version.
2023-05-17 03:44:05 Please configure Metricbeat to monitor Logstash. Documentation can be found at: 
2023-05-17 03:44:05 https://www.elastic.co/guide/en/logstash/current/monitoring-with-metricbeat.html
2023-05-17 03:44:06 [2023-05-16T22:14:06,198][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}}
2023-05-17 03:44:06 [2023-05-16T22:14:06,265][WARN ][logstash.licensechecker.licensereader] Restored connection to ES instance {:url=>"http://elasticsearch:9200/"}
2023-05-17 03:44:06 [2023-05-16T22:14:06,271][INFO ][logstash.licensechecker.licensereader] Elasticsearch version determined (7.4.0) {:es_version=>7}
2023-05-17 03:44:06 [2023-05-16T22:14:06,271][WARN ][logstash.licensechecker.licensereader] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
2023-05-17 03:44:06 [2023-05-16T22:14:06,311][INFO ][logstash.monitoring.internalpipelinesource] Monitoring License OK
2023-05-17 03:44:06 [2023-05-16T22:14:06,311][INFO ][logstash.monitoring.internalpipelinesource] Validated license for monitoring. Enabling monitoring pipeline.
2023-05-17 03:44:06 [2023-05-16T22:14:06,407][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
2023-05-17 03:44:06 [2023-05-16T22:14:06,694][INFO ][org.reflections.Reflections] Reflections took 86 ms to scan 1 urls, producing 132 keys and 462 values
2023-05-17 03:44:06 [2023-05-16T22:14:06,797][INFO ][logstash.javapipeline    ] Pipeline `.monitoring-logstash` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
2023-05-17 03:44:06 [2023-05-16T22:14:06,815][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearchMonitoring", :hosts=>["http://elasticsearch:9200"]}
2023-05-17 03:44:06 [2023-05-16T22:14:06,819][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}}
2023-05-17 03:44:06 [2023-05-16T22:14:06,837][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Restored connection to ES instance {:url=>"http://elasticsearch:9200/"}
2023-05-17 03:44:06 [2023-05-16T22:14:06,842][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch version determined (7.4.0) {:es_version=>7}
2023-05-17 03:44:06 [2023-05-16T22:14:06,842][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
2023-05-17 03:44:06 [2023-05-16T22:14:06,870][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
2023-05-17 03:44:06 [2023-05-16T22:14:06,871][WARN ][logstash.javapipeline    ][.monitoring-logstash] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
2023-05-17 03:44:06 [2023-05-16T22:14:06,880][INFO ][logstash.javapipeline    ][.monitoring-logstash] Starting pipeline {:pipeline_id=>".monitoring-logstash", "pipeline.workers"=>1, "pipeline.batch.size"=>2, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>2, "pipeline.sources"=>["monitoring pipeline"], :thread=>"#<Thread:0x1a735e6a@/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
2023-05-17 03:44:07 [2023-05-16T22:14:07,268][INFO ][logstash.javapipeline    ][.monitoring-logstash] Pipeline Java execution initialization time {"seconds"=>0.39}
2023-05-17 03:44:07 [2023-05-16T22:14:07,275][INFO ][logstash.javapipeline    ][.monitoring-logstash] Pipeline started {"pipeline.id"=>".monitoring-logstash"}
2023-05-17 03:44:07 [2023-05-16T22:14:07,290][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:".monitoring-logstash"], :non_running_pipelines=>[]}
2023-05-17 03:44:08 [2023-05-16T22:14:08,875][INFO ][logstash.javapipeline    ][.monitoring-logstash] Pipeline terminated {"pipeline.id"=>".monitoring-logstash"}
2023-05-17 03:44:09 [2023-05-16T22:14:09,329][INFO ][logstash.pipelinesregistry] Removed pipeline from registry successfully {:pipeline_id=>:".monitoring-logstash"}
2023-05-17 03:44:09 [2023-05-16T22:14:09,336][INFO ][logstash.runner          ] Logstash shut down.
2023-05-17 03:44:10 Using bundled JDK: /usr/share/logstash/jdk
2023-05-17 03:53:00 Using bundled JDK: /usr/share/logstash/jdk
2023-05-17 03:53:17 Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
2023-05-17 03:53:17 [2023-05-16T22:23:17,718][INFO ][logstash.runner          ] Log4j configuration path used is: /usr/share/logstash/config/log4j2.properties
2023-05-17 03:53:17 [2023-05-16T22:23:17,722][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.7.0", "jruby.version"=>"jruby 9.3.10.0 (2.6.8) 2023-02-01 107b2e6697 OpenJDK 64-Bit Server VM 17.0.6+10 on 17.0.6+10 +indy +jit [x86_64-linux]"}
2023-05-17 03:53:17 [2023-05-16T22:23:17,724][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Dls.cgroup.cpuacct.path.override=/, -Dls.cgroup.cpu.path.override=/, -Xmx256m, -Xms256m, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
2023-05-17 03:53:17 [2023-05-16T22:23:17,918][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
2023-05-17 03:53:18 [2023-05-16T22:23:18,363][WARN ][deprecation.logstash.monitoringextension.pipelineregisterhook] Internal collectors option for Logstash monitoring is deprecated and targeted for removal in the next major version.
2023-05-17 03:53:18 Please configure Metricbeat to monitor Logstash. Documentation can be found at: 
2023-05-17 03:53:18 https://www.elastic.co/guide/en/logstash/current/monitoring-with-metricbeat.html
2023-05-17 03:53:18 [2023-05-16T22:23:18,706][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}}
2023-05-17 03:53:18 [2023-05-16T22:23:18,863][WARN ][logstash.licensechecker.licensereader] Restored connection to ES instance {:url=>"http://elasticsearch:9200/"}
2023-05-17 03:53:18 [2023-05-16T22:23:18,870][INFO ][logstash.licensechecker.licensereader] Elasticsearch version determined (7.4.0) {:es_version=>7}
2023-05-17 03:53:18 [2023-05-16T22:23:18,870][WARN ][logstash.licensechecker.licensereader] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
2023-05-17 03:53:18 [2023-05-16T22:23:18,916][INFO ][logstash.monitoring.internalpipelinesource] Monitoring License OK
2023-05-17 03:53:18 [2023-05-16T22:23:18,916][INFO ][logstash.monitoring.internalpipelinesource] Validated license for monitoring. Enabling monitoring pipeline.
2023-05-17 03:53:18 [2023-05-16T22:23:18,985][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
2023-05-17 03:53:19 [2023-05-16T22:23:19,433][INFO ][org.reflections.Reflections] Reflections took 120 ms to scan 1 urls, producing 132 keys and 462 values
2023-05-17 03:53:19 [2023-05-16T22:23:19,554][INFO ][logstash.javapipeline    ] Pipeline `.monitoring-logstash` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
2023-05-17 03:53:19 [2023-05-16T22:23:19,576][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearchMonitoring", :hosts=>["http://elasticsearch:9200"]}
2023-05-17 03:53:19 [2023-05-16T22:23:19,583][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}}
2023-05-17 03:53:19 [2023-05-16T22:23:19,600][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Restored connection to ES instance {:url=>"http://elasticsearch:9200/"}
2023-05-17 03:53:19 [2023-05-16T22:23:19,606][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch version determined (7.4.0) {:es_version=>7}
2023-05-17 03:53:19 [2023-05-16T22:23:19,606][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
2023-05-17 03:53:19 [2023-05-16T22:23:19,652][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
2023-05-17 03:53:19 [2023-05-16T22:23:19,652][WARN ][logstash.javapipeline    ][.monitoring-logstash] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
2023-05-17 03:53:19 [2023-05-16T22:23:19,661][INFO ][logstash.javapipeline    ][.monitoring-logstash] Starting pipeline {:pipeline_id=>".monitoring-logstash", "pipeline.workers"=>1, "pipeline.batch.size"=>2, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>2, "pipeline.sources"=>["monitoring pipeline"], :thread=>"#<Thread:0x512565a5@/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
2023-05-17 03:53:20 [2023-05-16T22:23:20,168][INFO ][logstash.javapipeline    ][.monitoring-logstash] Pipeline Java execution initialization time {"seconds"=>0.51}
2023-05-17 03:53:20 [2023-05-16T22:23:20,179][INFO ][logstash.javapipeline    ][.monitoring-logstash] Pipeline started {"pipeline.id"=>".monitoring-logstash"}
2023-05-17 03:53:20 [2023-05-16T22:23:20,188][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:".monitoring-logstash"], :non_running_pipelines=>[]}
2023-05-17 03:53:21 [2023-05-16T22:23:21,654][INFO ][logstash.javapipeline    ][.monitoring-logstash] Pipeline terminated {"pipeline.id"=>".monitoring-logstash"}
2023-05-17 03:53:22 [2023-05-16T22:23:22,208][INFO ][logstash.pipelinesregistry] Removed pipeline from registry successfully {:pipeline_id=>:".monitoring-logstash"}
2023-05-17 03:53:22 [2023-05-16T22:23:22,212][INFO ][logstash.runner          ] Logstash shut down.

// in docker desktop , i just added config file in logstash.config in pipeline , and changed nothing else. it doesn't show any massive error , but doesn't start also.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.