# Docker compose ELK

**URL:** <https://discuss.elastic.co/t/docker-compose-elk/368923>\
**Category:** Discussions en français\
**Created:** [October 16, 2024, 2:13pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923 "2024-10-16T14:13:02Z")\
**Posts on this page:** 13\
**Page:** 3

<div class="post-metadata">

**Author:** ![mamadou\_yassaou\_bah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mamadou_yassaou_bah/32/136168_2.png) [@mamadou\_yassaou\_bah](https://discuss.elastic.co/u/mamadou_yassaou_bah)\
**Post date:** [October 25, 2024, 2:45pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/44 "2024-10-25T14:45:37Z")

</div>

j'ai arreter mon ELK qui etait en cours voici pour l'instant c'est bon 🙂

```auto
[billing@ogn-prebilling test]$ docker compose up -d
WARN[0000] /home/billing/test/docker-compose.yml: `version` is obsolete
[+] Running 4/4
 ✔ Network test_default Created 0.7s
 ✔ Container test-setup-1 Healthy 5.5s
 ✔ Container test-es01-1 Healthy 77.7s
 ✔ Container test-kibana-1 Started 78.2s
[billing@ogn-prebilling test]$
[billing@ogn-prebilling test]$
[billing@ogn-prebilling test]$
[billing@ogn-prebilling test]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2da58ccb49a8 docker.elastic.co/kibana/kibana:8.15.3 "/bin/tini -- /usr/l…" About an hour ago Up About an hour (healthy) 0.0.0.0:5601->5601/tcp, :::5601->5601/tcp test-kibana-1
5bd26eb2ba30 docker.elastic.co/elasticsearch/elasticsearch:8.15.3 "/bin/tini -- /usr/l…" About an hour ago Up About an hour (healthy) 0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp test-es01-1
fee3e7e5c5f4 docker.elastic.co/elasticsearch/elasticsearch:8.13.2 "/bin/tini -- /usr/l…" 4 days ago Up 4 days (healthy) 9200/tcp, 9300/tcp 8130-setup-1
[billing@ogn-prebilling test]$

```

---

<div class="post-metadata">

**Author:** ![mamadou\_yassaou\_bah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mamadou_yassaou_bah/32/136168_2.png) [@mamadou\_yassaou\_bah](https://discuss.elastic.co/u/mamadou_yassaou_bah)\
**Post date:** [October 25, 2024, 2:55pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/45 "2024-10-25T14:55:48Z")

</div>

AIDEZ S'Il VOUS PLAIT

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 25, 2024, 3:51pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/46 "2024-10-25T15:51:03Z")

</div>

Et donc c'est bon ?

Kibana s'ouvre maintenant ?  
Il te demande un login/password ?

---

<div class="post-metadata">

**Author:** ![mamadou\_yassaou\_bah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mamadou_yassaou_bah/32/136168_2.png) [@mamadou\_yassaou\_bah](https://discuss.elastic.co/u/mamadou_yassaou_bah)\
**Post date:** [October 25, 2024, 4:13pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/47 "2024-10-25T16:13:00Z")

</div>

oui il me demande le login/password

mais l'objectif c'est de l'apter a mon ancien ELK

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 25, 2024, 4:54pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/48 "2024-10-25T16:54:02Z")

</div>

Ha mince. Si j'avais su que tu avais déjà des données de prod, on serait peut-être parti sur une autre piste...

Je suppose que les deux clusters ne peuvent pas se voir si?

Il faudrait les mettre dans le même réseau Docker je pense et faire un reindex from remote.

Sinon plus simple, tu réindexes tout ton dataset comme tu l'avais fait initialement.

Si tu ne veux pas, il va falloir que tu stoppes ce qu'on a fait ensemble car finalement tu n'auras pas besoin de ce cluster neuf et conforme.  
Mais que tu appliques le guide ici : [Manually configure security | Elasticsearch Guide [8.15] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.15/manually-configure-security.html)

> If you want to enable security on an existing, unsecured cluster, use your own Certificate Authority (CA), or would rather manually configure security, the following scenarios provide steps for configuring TLS on the transport layer, plus securing HTTPS traffic if you want it.

Je te recommande ici de faire au moins Basic Security, c'est à dire ce guide: [Set up basic security for the Elastic Stack | Elasticsearch Guide [8.15] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.15/security-basic-setup.html)

Attention ! C'est pas simple du tout... Et c'est pour ça que je recommande de partir d'un cluster propre et neuf plutôt que d'essayer de rafistoler une configuration que tu as dégradée en mettant `xpack.security.enabled=false`.

---

<div class="post-metadata">

**Author:** ![mamadou\_yassaou\_bah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mamadou_yassaou_bah/32/136168_2.png) [@mamadou\_yassaou\_bah](https://discuss.elastic.co/u/mamadou_yassaou_bah)\
**Post date:** [October 29, 2024, 2:39pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/49 "2024-10-29T14:39:55Z")

</div>

comme tu me le conseiller je suis repartie sur une base saine

l'elastic et le kibana marche mais le logstash voici le docker compose

```auto
[billing@ogn-prebilling test]$ cat docker-compose.yml
version: "2.2"

services:
  setup:
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
    user: "0"
    command: >
      bash -c '
        if [x${ELASTIC_PASSWORD} == x]; then
          echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
          exit 1;
        elif [x${KIBANA_PASSWORD} == x]; then
          echo "Set the KIBANA_PASSWORD environment variable in the .env file";
          exit 1;
        fi;
        if [! -f config/certs/ca.zip]; then
          echo "Creating CA";
          bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
          unzip config/certs/ca.zip -d config/certs;
        fi;
        if [! -f config/certs/certs.zip]; then
          echo "Creating certs";
          echo -ne \
          "instances:\n"\
          " - name: es01\n"\
          " dns:\n"\
          " - es01\n"\
          " - localhost\n"\
          " ip:\n"\
          " - 127.0.0.1\n"\
          > config/certs/instances.yml;
          bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
          unzip config/certs/certs.zip -d config/certs;
        fi;
        echo "Setting file permissions"
        chown -R root:root config/certs;
        find . -type d -exec chmod 750 \{\} \;;
        find . -type f -exec chmod 640 \{\} \;;
        echo "Waiting for Elasticsearch availability";
        until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
        echo "Setting kibana_system password";
        until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
        echo "All done!";
      '
    healthcheck:
      test: ["CMD-SHELL", "[-f config/certs/es01/es01.crt]"]
      interval: 1s
      timeout: 5s
      retries: 120

  es01:
    depends_on:
      setup:
        condition: service_healthy
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - ${ES_PORT}:9200
    environment:
      - node.name=es01
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01
      - "ES_JAVA_OPTS=-Xms4g -Xmx4g"
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es01/es01.key
      - xpack.security.http.ssl.certificate=certs/es01/es01.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es01/es01.key
      - xpack.security.transport.ssl.certificate=certs/es01/es01.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
    #mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120
  logstash:
    image: docker.elastic.co/logstash/logstash:8.13.2
    container_name: logstash
    depends_on:
      es01:
        condition: service_healthy
    ports:
      - "5044:5044"
    environment:
      - xpack.monitoring.elasticsearch.hosts=https://es01:9200
      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
      - LS_JAVA_OPTS=-Xms8g -Xmx8g
    volumes:
      - certs:/usr/share/logstash/config/certs
      - ./logstash/pipeline:/usr/share/logstash/pipeline
    logging:
      driver: json-file
      options:
        max-size: "1g"
        max-file: "3"

  kibana:
    depends_on:
      es01:
        condition: service_healthy
    image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
    volumes:
      - certs:/usr/share/kibana/config/certs
      - kibanadata:/usr/share/kibana/data
    ports:
      - ${KIBANA_PORT}:5601
    environment:
      - SERVERNAME=kibana
      - ELASTICSEARCH_HOSTS=https://es01:9200
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
    # - I18N_LOCALE=fr-FR
    mem_limit: ${MEM_LIMIT}
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

volumes:
  certs:
    driver: local
  esdata01:
    driver: local
  kibanadata:

    driver: local
[billing@ogn-prebilling test]$

```

et voici l'erreur

```auto
[billing@ogn-prebilling test]$ docker logs logstash
2024/10/29 14:05:31 Setting 'xpack.monitoring.elasticsearch.hosts' from environment.
Using bundled JDK: /usr/share/logstash/jdk
/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/java_thread_pool_executor.rb:13: warning: method redefined; discarding old to_int
/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/java_thread_pool_executor.rb:13: warning: method redefined; discarding old to_f
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2024-10-29T14:05:47,756][INFO][logstash.runner] Log4j configuration path used is: /usr/share/logstash/config/log4j2.properties
[2024-10-29T14:05:47,761][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"8.13.2", "jruby.version"=>"jruby 9.4.5.0 (3.1.4) 2023-11-02 1abae2700f OpenJDK 64-Bit Server VM 17.0.10+7 on 17.0.10+7 +indy +jit [x86_64-linux]"}
[2024-10-29T14:05:47,764][INFO][logstash.runner] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Dlogstash.jackson.stream-read-constraints.max-string-length=200000000, -Dlogstash.jackson.stream-read-constraints.max-number-length=10000, -Dls.cgroup.cpuacct.path.override=/, -Dls.cgroup.cpu.path.override=/, -Xms8g, -Xmx8g, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED, -Dio.netty.allocator.maxOrder=11]
[2024-10-29T14:05:47,766][INFO][logstash.runner] Jackson default value override `logstash.jackson.stream-read-constraints.max-string-length` configured to `200000000`
[2024-10-29T14:05:47,766][INFO][logstash.runner] Jackson default value override `logstash.jackson.stream-read-constraints.max-number-length` configured to `10000`
[2024-10-29T14:05:47,773][INFO][logstash.settings] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
[2024-10-29T14:05:47,774][INFO][logstash.settings] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
[2024-10-29T14:05:47,968][INFO][logstash.agent] No persistent UUID file found. Generating new UUID {:uuid=>"6ca35812-f920-4411-8ae4-c292304dfa22", :path=>"/usr/share/logstash/data/uuid"}
[2024-10-29T14:05:48,533][WARN][logstash.monitoringextension.pipelineregisterhook] xpack.monitoring.enabled has not been defined, but found elasticsearch configuration. Please explicitly set `xpack.monitoring.enabled: true` in logstash.yml
[2024-10-29T14:05:48,535][WARN][deprecation.logstash.monitoringextension.pipelineregisterhook] Internal collectors option for Logstash monitoring is deprecated and targeted for removal in the next major version.
Please configure Elastic Agent to monitor Logstash. Documentation can be found at:
https://www.elastic.co/guide/en/logstash/current/monitoring-with-elastic-agent.html
[2024-10-29T14:05:48,939][INFO][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://es01:9200/]}}
[2024-10-29T14:05:49,172][INFO][logstash.licensechecker.licensereader] Failed to perform request {:message=>"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target", :exception=>Manticore::ClientProtocolException, :cause=>#<Java::JavaxNetSsl::SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target>}
[2024-10-29T14:05:49,173][WARN][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://es01:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [https://es01:9200/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}
[2024-10-29T14:05:49,228][INFO][logstash.licensechecker.licensereader] Failed to perform request {:message=>"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target", :exception=>Manticore::ClientProtocolException, :cause=>#<Java::JavaxNetSsl::SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target>}
[2024-10-29T14:05:49,229][WARN][logstash.licensechecker.licensereader] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [https://es01:9200/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target {:url=>https://es01:9200/, :error_message=>"Elasticsearch Unreachable: [https://es01:9200/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
[2024-10-29T14:05:49,234][WARN][logstash.licensechecker.licensereader] Attempt to fetch Elasticsearch cluster info failed. Sleeping for 0.02 {:fail_count=>1, :exception=>"Elasticsearch Unreachable: [https://es01:9200/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}
[2024-10-29T14:05:49,257][ERROR][logstash.licensechecker.licensereader] Unable to retrieve Elasticsearch cluster info. {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError}
[2024-10-29T14:05:49,259][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"No Available connections"}
[2024-10-29T14:05:49,274][ERROR][logstash.monitoring.internalpipelinesource] Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.
[2024-10-29T14:05:49,467][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2024-10-29T14:05:52,000][INFO][org.reflections.Reflections] Reflections took 100 ms to scan 1 urls, producing 132 keys and 468 values
[2024-10-29T14:05:53,868][ERROR][logstash.outputs.elasticsearch] Invalid setting for elasticsearch output plugin:

  output {
    elasticsearch {
      # This setting must be a path
      # ["File does not exist or cannot be opened ca.crt"]
      ssl_certificate_authorities => "ca.crt"
      ...
    }
  }
[2024-10-29T14:05:53,883][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration.", :backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:120)", "org.logstash.execution.AbstractPipelineExt.initialize(AbstractPipelineExt.java:186)", "org.logstash.execution.AbstractPipelineExt$INVOKER$i$initialize.call(AbstractPipelineExt$INVOKER$i$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:847)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1319)", "org.jruby.ir.instructions.InstanceSuperInstr.interpret(InstanceSuperInstr.java:139)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:367)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:128)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:115)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:446)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:92)", "org.jruby.RubyClass.newInstance(RubyClass.java:931)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:446)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:92)", "org.jruby.ir.instructions.CallBase.interpret(CallBase.java:548)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:367)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)", "org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:88)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:238)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:225)", "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:228)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:476)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:293)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:328)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)", "org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:116)", "org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:136)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:66)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.Block.call(Block.java:144)", "org.jruby.RubyProc.call(RubyProc.java:352)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:111)", "java.base/java.lang.Thread.run(Thread.java:840)"]}
[2024-10-29T14:05:53,901][INFO][logstash.runner] Logstash shut down.
[2024-10-29T14:05:53,909][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:808) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:767) ~[jruby.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:90) ~[?:?]
[billing@ogn-prebilling test]$

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 29, 2024, 3:09pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/50 "2024-10-29T15:09:21Z")

</div>

C'est quoi ton fichier de config du job logstash?

---

<div class="post-metadata">

**Author:** ![mamadou\_yassaou\_bah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mamadou_yassaou_bah/32/136168_2.png) [@mamadou\_yassaou\_bah](https://discuss.elastic.co/u/mamadou_yassaou_bah)\
**Post date:** [October 29, 2024, 3:13pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/51 "2024-10-29T15:13:18Z")

</div>

voici mon pipeline.config j'ai enlver le haut car c'est sensible

```auto
.
.
}

filter {
    if "audit_processing_out" in [tags] {
        ruby {
            code => "
                event.set('document_id', event.get('file_id').to_s + '_' + event.get('target').to_s)
            "
        }
    }

    if "audit_processing_in" in [tags] {
        ruby {
            code => "
                event.set('document_id', event.get('file_id').to_s + '_' + event.get('target').to_s)
            "
        }
    }

    if "audit_forwarding" in [tags] {
        ruby {
            code => "
                event.set('document_id', event.get('file_id').to_s + '_' + event.get('target').to_s)
            "
        }
    }
}

output {
    if "audit_collect" in [tags] {
        elasticsearch {
            hosts => ["https://es01:9200"]
            index => "statficcollected"
            data_stream => false
            document_id => "%{file_id}"
            user => "elastic"
            password => "changeme"
            ssl_enabled => true
            ssl_certificate_authorities => "ca.crt"
        }
    }

    if "audit_processing_out" in [tags] {
        elasticsearch {
            hosts => ["https://es01:9200"]
            index => "statficprocedout"
            data_stream => false
            document_id => "%{document_id}"
            user => "elastic"
            password => "changeme"
            ssl_enabled => true
            ssl_certificate_authorities => "ca.crt"
        }
    }

    if "audit_processing_in" in [tags] {
        elasticsearch {
            hosts => ["https://es01:9200"]
            index => "statficprocedin"
            data_stream => false
            document_id => "%{document_id}"
            user => "elastic"
            password => "changeme"
            ssl_enabled => true
            ssl_certificate_authorities => "ca.crt"
        }
    }

    if "audit_forwarding" in [tags] {
        elasticsearch {
            hosts => ["https://es01:9200"]
            index => "statficforward"
            data_stream => false
            document_id => "%{document_id}"
            user => "elastic"
            password => "changeme"
            ssl_enabled => true
            ssl_certificate_authorities => "ca.crt"
        }
    }

    stdout { codec => json }
}
[billing@ogn-prebilling test]$

```

c'est ce ficher qui envoi les donner

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 29, 2024, 3:30pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/52 "2024-10-29T15:30:22Z")

</div>

Visiblement cette ligne n'est pas bonne:

```
ssl_certificate_authorities => "ca.crt"

```

Puisque tu as déjà passé le crt dans le docker compose, je pense que c'est inutile ici.

---

<div class="post-metadata">

**Author:** ![mamadou\_yassaou\_bah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mamadou_yassaou_bah/32/136168_2.png) [@mamadou\_yassaou\_bah](https://discuss.elastic.co/u/mamadou_yassaou_bah)\
**Post date:** [October 29, 2024, 3:33pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/53 "2024-10-29T15:33:48Z")

</div>

donc je dois l'enlever dans le logstash si je comprend bien ?

---

<div class="post-metadata">

**Author:** ![mamadou\_yassaou\_bah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mamadou_yassaou_bah/32/136168_2.png) [@mamadou\_yassaou\_bah](https://discuss.elastic.co/u/mamadou_yassaou_bah)\
**Post date:** [October 29, 2024, 3:40pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/54 "2024-10-29T15:40:01Z")

</div>

j'ai enlever mais sa donne ça

```auto
[billing@ogn-prebilling test]$ docker logs logstash
2024/10/29 15:36:37 Setting 'xpack.monitoring.enabled' from environment.
2024/10/29 15:36:37 Setting 'xpack.monitoring.elasticsearch.hosts' from environment.
Using bundled JDK: /usr/share/logstash/jdk
/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/java_thread_pool_executor.rb:13: warning: method redefined; discarding old to_int
/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/java_thread_pool_executor.rb:13: warning: method redefined; discarding old to_f
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2024-10-29T15:36:54,845][INFO][logstash.runner] Log4j configuration path used is: /usr/share/logstash/config/log4j2.properties
[2024-10-29T15:36:54,851][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"8.13.2", "jruby.version"=>"jruby 9.4.5.0 (3.1.4) 2023-11-02 1abae2700f OpenJDK 64-Bit Server VM 17.0.10+7 on 17.0.10+7 +indy +jit [x86_64-linux]"}
[2024-10-29T15:36:54,853][INFO][logstash.runner] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Dlogstash.jackson.stream-read-constraints.max-string-length=200000000, -Dlogstash.jackson.stream-read-constraints.max-number-length=10000, -Dls.cgroup.cpuacct.path.override=/, -Dls.cgroup.cpu.path.override=/, -Xms8g, -Xmx8g, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED, -Dio.netty.allocator.maxOrder=11]
[2024-10-29T15:36:54,855][INFO][logstash.runner] Jackson default value override `logstash.jackson.stream-read-constraints.max-string-length` configured to `200000000`
[2024-10-29T15:36:54,855][INFO][logstash.runner] Jackson default value override `logstash.jackson.stream-read-constraints.max-number-length` configured to `10000`
[2024-10-29T15:36:54,861][INFO][logstash.settings] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
[2024-10-29T15:36:54,862][INFO][logstash.settings] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
[2024-10-29T15:36:55,156][INFO][logstash.agent] No persistent UUID file found. Generating new UUID {:uuid=>"dc301467-27f5-4222-87fa-f177242d51db", :path=>"/usr/share/logstash/data/uuid"}
[2024-10-29T15:36:55,931][WARN][deprecation.logstash.monitoringextension.pipelineregisterhook] Internal collectors option for Logstash monitoring is deprecated and targeted for removal in the next major version.
Please configure Elastic Agent to monitor Logstash. Documentation can be found at:
https://www.elastic.co/guide/en/logstash/current/monitoring-with-elastic-agent.html
[2024-10-29T15:36:56,241][INFO][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://es01:9200/]}}
[2024-10-29T15:36:56,454][INFO][logstash.licensechecker.licensereader] Failed to perform request {:message=>"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target", :exception=>Manticore::ClientProtocolException, :cause=>#<Java::JavaxNetSsl::SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target>}
[2024-10-29T15:36:56,455][WARN][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://es01:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [https://es01:9200/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}
[2024-10-29T15:36:56,489][INFO][logstash.licensechecker.licensereader] Failed to perform request {:message=>"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target", :exception=>Manticore::ClientProtocolException, :cause=>#<Java::JavaxNetSsl::SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target>}
[2024-10-29T15:36:56,490][WARN][logstash.licensechecker.licensereader] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [https://es01:9200/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target {:url=>https://es01:9200/, :error_message=>"Elasticsearch Unreachable: [https://es01:9200/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
[2024-10-29T15:36:56,492][WARN][logstash.licensechecker.licensereader] Attempt to fetch Elasticsearch cluster info failed. Sleeping for 0.02 {:fail_count=>1, :exception=>"Elasticsearch Unreachable: [https://es01:9200/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}
[2024-10-29T15:36:56,520][ERROR][logstash.licensechecker.licensereader] Unable to retrieve Elasticsearch cluster info. {:message=>"No Available connections", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError}
[2024-10-29T15:36:56,521][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"No Available connections"}
[2024-10-29T15:36:56,543][ERROR][logstash.monitoring.internalpipelinesource] Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.
[2024-10-29T15:36:56,695][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [\\t\\r\\n], \"#\", \"input\", \"filter\", \"output\" at line 1, column 1 (byte 1)", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:239:in `initialize'", "org/logstash/execution/AbstractPipelineExt.java:173:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48:in `initialize'", "org/jruby/RubyClass.java:931:in `new'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:49:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:386:in `block in converge_state'"]}
[2024-10-29T15:36:56,728][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2024-10-29T15:36:56,760][INFO][logstash.runner] Logstash shut down.
[2024-10-29T15:36:56,771][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:808) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:767) ~[jruby.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:90) ~[?:?]
[billing@ogn-prebilling test]$
[billing@ogn-prebilling test]$
[billing@ogn-prebilling test]$
[billing@ogn-prebilling test]$

```

---

<div class="post-metadata">

**Author:** ![mamadou\_yassaou\_bah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mamadou_yassaou_bah/32/136168_2.png) [@mamadou\_yassaou\_bah](https://discuss.elastic.co/u/mamadou_yassaou_bah)\
**Post date:** [October 29, 2024, 4:14pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/55 "2024-10-29T16:14:51Z")

</div>

je pense que le probleme se trouve au niveau de la communication en logstash et elasticsearch

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 29, 2024, 8:39pm UTC](https://discuss.elastic.co/t/docker-compose-elk/368923/56 "2024-10-29T20:39:00Z")

</div>

Oui c'est toujours le pb avec des certificats auto générés.  
Il n'y a pas ce pb avec de vrais certificats (genre ceux que tu auras sur [cloud.elastic.co](http://cloud.elastic.co) ou si tu fournis des certificats SSL avec [https://letsencrypt.org/](https://letsencrypt.org/) par exemple).

Là, il faut donner plus d'info.

Peut-être qu'il faut alors mettre:

```auto
ssl_certificate_authorities => "/usr/share/logstash/config/certs/ca/ca.crt"

```

Quelque chose comme ça...

[Previous page](https://discuss.elastic.co/t/docker-compose-elk/368923.md?page=2)