Elastic-agent > nginx > logstash -- nginx proxy error

marlonws · April 29, 2024, 2:43am

I configured the fleet server perfectly, the policy that makes the elastic-agent output directly to elasticsearch works normally.

I configured a logstash, without errors, it connects normally to elasticsearch. Logstash is behind nginx and when I configure elastic-agent to send data to logstash, nginx is giving error 400 and nothing arrives in the logstash logs.

logstash.conf

input {
  elastic_agent {
    port => 5045
    ssl_enabled => true
    ssl_certificate_authorities => "/etc/logstash/certs/ca-logstash.crt"
    ssl_certificate => "/etc/logstash/certs/cert-logstash.crt"
    ssl_key => "/etc/logstash/certs/key-logstash.crt"
    ssl_client_authentication => "optional"
  }
}

output {
  elasticsearch {
    hosts => ["https://elastic.xxxxxxxx:443"]
      manage_template => false
      data_stream => auto
      index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
      ssl_enabled => true
      ssl_verification_mode => none
      ssl_certificate_authorities => "/etc/logstash/certs/ca-coord4.cer"
      api_key => "xxxxxxxx:xxxxxx"
    }
}

nginx conf.d

server {

        listen 443 ssl default_server;

        ssl_certificate /etc/logstash/certs/cert-logstash.crt;
        ssl_certificate_key /etc/logstash/certs/key-logstash.crt;
        ssl_trusted_certificate /etc/logstash/certs/ca-logstash.crt;

        server_name logstash.xxxxxxx.com.br;

        location / {
                    proxy_ignore_client_abort on;
                    proxy_pass https://localhost:5045;
                    proxy_set_header  X-Real-IP  $remote_addr;
                    proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
                    proxy_set_header X-Forwarded-Protocol $scheme;
                    proxy_set_header  Host $host;
        }

The fleet server is normal, to test I used an elastic-agent that is not already from the fleetserver itself.

In the nginx logs I see the elastic-agent ip trying to send the logs to nginx, but nginx gives this encoded error:

189.68.xx.xx - - [29/Apr/2024:02:08:45 +0000] "2W\x00\x00\x00\x022C\x00\x00\x08\xF8x^\xCCX\xCD\x8E\xE4\xB6\x11\x9E\xE4\x9C\xBC\xC3\xA0Nv@\xA9I\x8A\xFA\xE3\xC9\x8E\x93\x1C\x82\xC0\x17;\x97\x18\xC6\x80\x92\xA8ne%Q!\xA9\xD9\x1D7\xF4(y\xC4\xBCC\x00\x92\xFA\xE9\xD9\x19\xEF\xAE\x0Fk\xA3/-\x89,\x16\xEB\xF7\xFB\x8A\xFE\xFD\xEE\xEE\xEEwww\x7F\xF8\xDF\x15\xBE\xB2\xDD \x8D\x15\xC3\x04\x1C(\xA6,\xC2,\xA2\xE5\xF7\x98r\x8C9-b\xCA\xF0\xBF\x00\xC1W\x83\xB4\xA2\x11V\x00\xBFB%\x85\x05\x0Em\xD7K\xF7\x17\x81}\x9A$pxhT" 400 166 "-" "-"

nginx logs acess error 400:

marlonws · April 29, 2024, 2:46am

Just to talk more about the architecture:

Elasticsearch and kibana are in a cluster behind an Azure application gateway. Due to the difficulty of exposing logstash on https because of the logstash monitoring api, I decided to place logstash in a VM.

leandrojmp · April 29, 2024, 3:35am

This will not work, the elastic_agent or beats input in logstash does not use http or https, it uses a custom protocol over TCP.

If you want to have NGINX in front of Logstash you need to configure it to use tcp and not http.

Not sure how you do that on NGINX because I do not use it, I use HAProxy in front of Logstash.

For NGINX it seems that you need something like what is described here: TCP and UDP Load Balancing | NGINX Documentation

marlonws · April 29, 2024, 5:06am

Could you give me some light on how you do this with HAProxy?

Just a good example or some documentation.

leandrojmp · April 29, 2024, 5:19am

You can check the HAProxy documentation, like this one.

Basically you use mode tcp to tell that your frontend is listening on tcp, you will also need to look the documentation on how to configure ssl or how to make it to be a tcp pass-through.

But those things are out of the scope of these forum.

Rios · April 29, 2024, 6:04am

Also the question: have you set ssl_key in PKCS8 format?

marlonws · April 29, 2024, 6:26am

the keys are in ".pem" format in the logstash output in kibana

In the logstash application I left it as ".crt" which is practically a '.pem"

marlonws · April 29, 2024, 6:29am

From the link you sent, I believe my keys are PKCS1, I will check correctly.

marlonws · April 29, 2024, 6:56am

I changed it to PKCS8 format and changed it in kibana, logstash and nginx, but the strange error in nginx persists. I believe this is what the friend above said, to try to use the TPC/TLS protocol.

[2024-04-29T06:54:16,834][INFO ][logstash.runner          ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2024-04-29T06:54:16,839][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.13.2", "jruby.version"=>"jruby 9.4.5.0 (3.1.4) 2023-11-02 1abae2700f OpenJDK 64-Bit Server VM 17.0.10+7 on 17.0.10+7 +indy +jit [x86_64-linux]"}
[2024-04-29T06:54:16,846][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Dlogstash.jackson.stream-read-constraints.max-string-length=200000000, -Dlogstash.jackson.stream-read-constraints.max-number-length=10000, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED, -Dio.netty.allocator.maxOrder=11]
[2024-04-29T06:54:16,849][INFO ][logstash.runner          ] Jackson default value override `logstash.jackson.stream-read-constraints.max-string-length` configured to `200000000`
[2024-04-29T06:54:16,850][INFO ][logstash.runner          ] Jackson default value override `logstash.jackson.stream-read-constraints.max-number-length` configured to `10000`
[2024-04-29T06:54:17,678][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2024-04-29T06:54:18,194][INFO ][org.reflections.Reflections] Reflections took 114 ms to scan 1 urls, producing 132 keys and 468 values
[2024-04-29T06:54:18,547][INFO ][logstash.javapipeline    ] Pipeline `elastic-agent-pipeline` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2024-04-29T06:54:18,572][INFO ][logstash.outputs.elasticsearch][elastic-agent-pipeline] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://elastic.xxxxxxxx.com.br:443"]}
[2024-04-29T06:54:18,577][WARN ][logstash.outputs.elasticsearch][elastic-agent-pipeline] You have enabled encryption but DISABLED certificate verification, to make sure your data is secure set `ssl_verification_mode => full`
[2024-04-29T06:54:18,667][INFO ][logstash.outputs.elasticsearch][elastic-agent-pipeline] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elastic.xxxxx.com.br:443/]}}
[2024-04-29T06:54:18,985][WARN ][logstash.outputs.elasticsearch][elastic-agent-pipeline] Restored connection to ES instance {:url=>"https://elastic.xxxx.com.br:443/"}
[2024-04-29T06:54:18,986][INFO ][logstash.outputs.elasticsearch][elastic-agent-pipeline] Elasticsearch version determined (8.13.2) {:es_version=>8}
[2024-04-29T06:54:18,987][WARN ][logstash.outputs.elasticsearch][elastic-agent-pipeline] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2024-04-29T06:54:19,003][INFO ][logstash.outputs.elasticsearch][elastic-agent-pipeline] Not eligible for data streams because config contains one or more settings that are not compatible with data streams: {"data_stream"=>"auto", "index"=>"%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"}
[2024-04-29T06:54:19,004][INFO ][logstash.outputs.elasticsearch][elastic-agent-pipeline] Data streams auto configuration (`data_stream => auto` or unset) resolved to `false`
[2024-04-29T06:54:19,027][INFO ][logstash.javapipeline    ][elastic-agent-pipeline] Starting pipeline {:pipeline_id=>"elastic-agent-pipeline", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, "pipeline.sources"=>["/etc/logstash/conf.d/logstash.conf"], :thread=>"#<Thread:0x267a9b85 /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2024-04-29T06:54:19,445][INFO ][logstash.javapipeline    ][elastic-agent-pipeline] Pipeline Java execution initialization time {"seconds"=>0.41}
[2024-04-29T06:54:19,451][INFO ][logstash.inputs.beats    ][elastic-agent-pipeline] Starting input listener {:address=>"0.0.0.0:5045"}
[2024-04-29T06:54:19,712][INFO ][logstash.javapipeline    ][elastic-agent-pipeline] Pipeline started {"pipeline.id"=>"elastic-agent-pipeline"}
[2024-04-29T06:54:19,723][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:"elastic-agent-pipeline"], :non_running_pipelines=>[]}
[2024-04-29T06:54:19,754][INFO ][org.logstash.beats.Server][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] Starting server on port: 5045

From the logstash log, it started normally and is talking to elasticsearch.

the problem is what the nginx protocol should be

Rios · April 29, 2024, 8:17am

Check what is on LS side:

input {
  elastic_agent {
    ssl_key => "/etc/logstash/certs/key-logstash.crt"

marlonws · April 29, 2024, 6:52pm

Yes I changed both sides. PKCS8 on kibana and vm logstash.

input {
  elastic_agent {
    port => 5045
    ssl_enabled => true
    ssl_certificate_authorities => "/etc/logstash/certs/ca-logstash.crt"
    ssl_certificate => "/etc/logstash/certs/cert-logstash.crt"
    ssl_key => "/etc/logstash/certs/key-logstash-pkcs8.key"
    ssl_client_authentication => "optional"
  }
}

Rios · April 29, 2024, 6:55pm

If still is not working, enable log.level = debug or trace, to see more details.

marlonws · April 29, 2024, 7:01pm

I don't get error on: /var/log/nginx/error.log , so it shouldn't be a handshake certificate error...

On /var/log/nginx/access.log , i get:

It must be the protocol that beats / elastic-agent only operates on TPC, no?

marlonws · April 29, 2024, 7:10pm

The log I got with the debug:

[2024-04-29T19:04:22,016][DEBUG][io.netty.handler.ssl.SslHandler][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] [id: 0x0948de65, L:/10.17.5.4:5045 - R:/4.255.50.155:2048] HANDSHAKEN: protocol:TLSv1.2 cipher suite:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
[2024-04-29T19:04:22,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:04:24,344][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:04:24,346][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:04:27,059][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:04:29,367][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:04:29,371][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:04:32,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:04:33,296][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@id = "plain_698b1c47-ddf1-4643-b99d-353c629d0733"
[2024-04-29T19:04:33,302][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@enable_metric = true
[2024-04-29T19:04:33,305][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@charset = "UTF-8"
[2024-04-29T19:04:33,332][DEBUG][io.netty.handler.ssl.SslHandler][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] [id: 0xcf5fad6f, L:/10.17.5.4:5045 - R:/4.255.50.155:1024] HANDSHAKEN: protocol:TLSv1.2 cipher suite:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
[2024-04-29T19:04:34,382][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:04:34,384][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:04:37,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:04:39,396][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:04:39,398][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:04:42,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:04:44,380][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1032037926} forced-compaction result (captures: `3` span: `PT10.006400612S`)
[2024-04-29T19:04:44,383][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1837242008} forced-compaction result (captures: `3` span: `PT10.010640599S`)
[2024-04-29T19:04:44,383][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1682421404} forced-compaction result (captures: `3` span: `PT10.011149909S`)
[2024-04-29T19:04:44,383][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=434811073} forced-compaction result (captures: `3` span: `PT10.011493616S`)
[2024-04-29T19:04:44,384][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1702793923} forced-compaction result (captures: `3` span: `PT10.011824624S`)
[2024-04-29T19:04:44,411][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:04:44,413][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:04:47,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:04:49,387][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1959327576} forced-compaction result (captures: `3` span: `PT10.011938201S`)
[2024-04-29T19:04:49,388][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=906552317} forced-compaction result (captures: `3` span: `PT10.012940524S`)
[2024-04-29T19:04:49,389][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=2127032847} forced-compaction result (captures: `3` span: `PT10.013120929S`)
[2024-04-29T19:04:49,389][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=949331810} forced-compaction result (captures: `3` span: `PT10.013215731S`)
[2024-04-29T19:04:49,389][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=190113055} forced-compaction result (captures: `3` span: `PT10.013297033S`)
[2024-04-29T19:04:49,389][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=2058545283} forced-compaction result (captures: `3` span: `PT10.013444736S`)
[2024-04-29T19:04:49,389][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1000618576} forced-compaction result (captures: `3` span: `PT10.01357544S`)
[2024-04-29T19:04:49,389][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=782181617} forced-compaction result (captures: `3` span: `PT10.013672342S`)
[2024-04-29T19:04:49,389][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=47140896} forced-compaction result (captures: `3` span: `PT10.013751743S`)
[2024-04-29T19:04:49,421][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:04:49,425][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:04:51,924][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@id = "plain_698b1c47-ddf1-4643-b99d-353c629d0733"
[2024-04-29T19:04:51,929][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@enable_metric = true
[2024-04-29T19:04:51,930][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@charset = "UTF-8"
[2024-04-29T19:04:51,950][DEBUG][io.netty.handler.ssl.SslHandler][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] [id: 0x703e7491, L:/10.17.5.4:5045 - R:/4.255.50.155:2048] HANDSHAKEN: protocol:TLSv1.2 cipher suite:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
[2024-04-29T19:04:52,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:04:54,440][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:04:54,444][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:04:57,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:04:59,456][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:04:59,459][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:05:02,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:05:03,301][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@id = "plain_698b1c47-ddf1-4643-b99d-353c629d0733"
[2024-04-29T19:05:03,301][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@enable_metric = true
[2024-04-29T19:05:03,303][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@charset = "UTF-8"
[2024-04-29T19:05:03,324][DEBUG][io.netty.handler.ssl.SslHandler][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] [id: 0x200a3e2a, L:/10.17.5.4:5045 - R:/4.255.50.155:1024] HANDSHAKEN: protocol:TLSv1.2 cipher suite:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
[2024-04-29T19:05:04,466][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:05:04,467][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:05:07,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:05:09,478][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:05:09,479][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:05:12,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:05:14,411][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1032037926} forced-compaction result (captures: `3` span: `PT10.005397002S`)
[2024-04-29T19:05:14,412][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1837242008} forced-compaction result (captures: `3` span: `PT10.00655743S`)
[2024-04-29T19:05:14,413][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1682421404} forced-compaction result (captures: `3` span: `PT10.00700104S`)
[2024-04-29T19:05:14,413][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=434811073} forced-compaction result (captures: `3` span: `PT10.007366348S`)
[2024-04-29T19:05:14,413][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1702793923} forced-compaction result (captures: `3` span: `PT10.00787596S`)
[2024-04-29T19:05:14,487][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:05:14,488][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:05:17,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:05:19,417][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1959327576} forced-compaction result (captures: `3` span: `PT10.008632282S`)
[2024-04-29T19:05:19,418][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=906552317} forced-compaction result (captures: `3` span: `PT10.009871311S`)
[2024-04-29T19:05:19,419][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=2127032847} forced-compaction result (captures: `3` span: `PT10.010371923S`)
[2024-04-29T19:05:19,419][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=949331810} forced-compaction result (captures: `3` span: `PT10.01069773S`)
[2024-04-29T19:05:19,420][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=190113055} forced-compaction result (captures: `3` span: `PT10.010972537S`)
[2024-04-29T19:05:19,420][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=2058545283} forced-compaction result (captures: `3` span: `PT10.011262243S`)
[2024-04-29T19:05:19,420][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1000618576} forced-compaction result (captures: `3` span: `PT10.011532749S`)
[2024-04-29T19:05:19,420][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=782181617} forced-compaction result (captures: `3` span: `PT10.011807556S`)
[2024-04-29T19:05:19,421][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=47140896} forced-compaction result (captures: `3` span: `PT10.012098963S`)
[2024-04-29T19:05:19,498][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:05:19,498][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:05:21,922][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@id = "plain_698b1c47-ddf1-4643-b99d-353c629d0733"
[2024-04-29T19:05:21,923][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@enable_metric = true
[2024-04-29T19:05:21,923][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@charset = "UTF-8"
[2024-04-29T19:05:21,936][DEBUG][io.netty.handler.ssl.SslHandler][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] [id: 0xd0fd7393, L:/10.17.5.4:5045 - R:/4.255.50.155:2048] HANDSHAKEN: protocol:TLSv1.2 cipher suite:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
[2024-04-29T19:05:22,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:05:24,505][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:05:24,506][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:05:27,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:05:29,519][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:05:29,520][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:05:32,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:05:33,292][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@id = "plain_698b1c47-ddf1-4643-b99d-353c629d0733"
[2024-04-29T19:05:33,293][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@enable_metric = true
[2024-04-29T19:05:33,293][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@charset = "UTF-8"
[2024-04-29T19:05:33,303][DEBUG][io.netty.handler.ssl.SslHandler][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] [id: 0x78cd1503, L:/10.17.5.4:5045 - R:/4.255.50.155:1024] HANDSHAKEN: protocol:TLSv1.2 cipher suite:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
[2024-04-29T19:05:34,430][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1321416374} forced-compaction result (captures: `13` span: `PT1M0.05816955S`)
[2024-04-29T19:05:34,430][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1284178367} forced-compaction result (captures: `13` span: `PT1M0.05856736S`)
[2024-04-29T19:05:34,431][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1993045099} forced-compaction result (captures: `13` span: `PT1M0.059372978S`)
[2024-04-29T19:05:34,432][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1020549899} forced-compaction result (captures: `13` span: `PT1M0.059649985S`)
[2024-04-29T19:05:34,432][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1802874017} forced-compaction result (captures: `13` span: `PT1M0.060111496S`)
[2024-04-29T19:05:34,526][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:05:34,527][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:05:37,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:05:39,436][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=2008321526} forced-compaction result (captures: `13` span: `PT1M0.06035866S`)
[2024-04-29T19:05:39,437][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=2134367096} forced-compaction result (captures: `13` span: `PT1M0.061459486S`)
[2024-04-29T19:05:39,441][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1266739991} forced-compaction result (captures: `13` span: `PT1M0.065531081S`)
[2024-04-29T19:05:39,441][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=2019351660} forced-compaction result (captures: `13` span: `PT1M0.065828788S`)
[2024-04-29T19:05:39,442][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=735327660} forced-compaction result (captures: `13` span: `PT1M0.065979492S`)
[2024-04-29T19:05:39,442][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1794754485} forced-compaction result (captures: `13` span: `PT1M0.066089794S`)
[2024-04-29T19:05:39,442][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=653087150} forced-compaction result (captures: `13` span: `PT1M0.066214897S`)
[2024-04-29T19:05:39,442][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=952915619} forced-compaction result (captures: `13` span: `PT1M0.0663418S`)
[2024-04-29T19:05:39,442][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=843017539} forced-compaction result (captures: `13` span: `PT1M0.066491303S`)
[2024-04-29T19:05:39,534][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:05:39,535][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:05:42,059][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:05:44,444][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1032037926} forced-compaction result (captures: `3` span: `PT10.014464666S`)
[2024-04-29T19:05:44,445][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1837242008} forced-compaction result (captures: `3` span: `PT10.014969878S`)
[2024-04-29T19:05:44,446][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1682421404} forced-compaction result (captures: `3` span: `PT10.014309963S`)
[2024-04-29T19:05:44,446][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=434811073} forced-compaction result (captures: `3` span: `PT10.014095258S`)
[2024-04-29T19:05:44,446][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1702793923} forced-compaction result (captures: `3` span: `PT10.013681548S`)
[2024-04-29T19:05:44,542][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:05:44,543][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:05:47,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:05:49,449][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1959327576} forced-compaction result (captures: `3` span: `PT10.012950612S`)
[2024-04-29T19:05:49,449][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=906552317} forced-compaction result (captures: `3` span: `PT10.012200395S`)
[2024-04-29T19:05:49,449][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=2127032847} forced-compaction result (captures: `3` span: `PT10.008252103S`)
[2024-04-29T19:05:49,450][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=949331810} forced-compaction result (captures: `3` span: `PT10.00883042S`)
[2024-04-29T19:05:49,451][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=190113055} forced-compaction result (captures: `3` span: `PT10.009138229S`)
[2024-04-29T19:05:49,451][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=2058545283} forced-compaction result (captures: `3` span: `PT10.009335535S`)
[2024-04-29T19:05:49,451][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1000618576} forced-compaction result (captures: `3` span: `PT10.009424438S`)
[2024-04-29T19:05:49,451][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=782181617} forced-compaction result (captures: `3` span: `PT10.00946534S`)
[2024-04-29T19:05:49,452][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=47140896} forced-compaction result (captures: `3` span: `PT10.009416139S`)
[2024-04-29T19:05:49,550][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:05:49,550][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:05:51,922][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@id = "plain_698b1c47-ddf1-4643-b99d-353c629d0733"
[2024-04-29T19:05:51,925][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@enable_metric = true
[2024-04-29T19:05:51,925][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@charset = "UTF-8"
[2024-04-29T19:05:51,934][DEBUG][io.netty.handler.ssl.SslHandler][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] [id: 0xcc53150d, L:/10.17.5.4:5045 - R:/4.255.50.155:2048] HANDSHAKEN: protocol:TLSv1.2 cipher suite:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
[2024-04-29T19:05:52,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:05:54,557][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:05:54,558][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:05:57,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:05:59,564][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:05:59,564][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:06:02,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:06:03,293][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@id = "plain_698b1c47-ddf1-4643-b99d-353c629d0733"
[2024-04-29T19:06:03,303][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@enable_metric = true
[2024-04-29T19:06:03,305][DEBUG][io.netty.handler.ssl.SslHandler][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] [id: 0xe75b4b90, L:/10.17.5.4:5045 - R:/4.255.50.155:1024] HANDSHAKEN: protocol:TLSv1.2 cipher suite:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
[2024-04-29T19:06:03,305][DEBUG][logstash.codecs.plain    ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@charset = "UTF-8"
[2024-04-29T19:06:04,459][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1321416374} forced-compaction result (captures: `13` span: `PT1M0.053243414S`)
[2024-04-29T19:06:04,459][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1284178367} forced-compaction result (captures: `13` span: `PT1M0.053570123S`)
[2024-04-29T19:06:04,459][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1993045099} forced-compaction result (captures: `13` span: `PT1M0.053669825S`)
[2024-04-29T19:06:04,459][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1020549899} forced-compaction result (captures: `13` span: `PT1M0.053748926S`)
[2024-04-29T19:06:04,459][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1802874017} forced-compaction result (captures: `13` span: `PT1M0.053822528S`)
[2024-04-29T19:06:04,570][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}
[2024-04-29T19:06:04,571][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}
[2024-04-29T19:06:07,058][DEBUG][org.logstash.execution.PeriodicFlush][elastic-agent-pipeline] Pushing flush onto pipeline.
[2024-04-29T19:06:09,462][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=2008321526} forced-compaction result (captures: `13` span: `PT1M0.053791827S`)
[2024-04-29T19:06:09,463][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=2134367096} forced-compaction result (captures: `13` span: `PT1M0.054084834S`)
[2024-04-29T19:06:09,463][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1266739991} forced-compaction result (captures: `13` span: `PT1M0.054167836S`)
[2024-04-29T19:06:09,463][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=2019351660} forced-compaction result (captures: `13` span: `PT1M0.054236338S`)
[2024-04-29T19:06:09,463][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=735327660} forced-compaction result (captures: `13` span: `PT1M0.05429204S`)
[2024-04-29T19:06:09,463][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=1794754485} forced-compaction result (captures: `13` span: `PT1M0.05434664S`)
[2024-04-29T19:06:09,463][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=653087150} forced-compaction result (captures: `13` span: `PT1M0.054399842S`)
[2024-04-29T19:06:09,463][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=952915619} forced-compaction result (captures: `13` span: `PT1M0.054450843S`)
[2024-04-29T19:06:09,463][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=last_1_minute id=843017539} forced-compaction result (captures: `13` span: `PT1M0.054502445S`)

marlonws · April 29, 2024, 7:14pm

Can you have any basis with these logs?

leandrojmp · April 29, 2024, 7:18pm

Did you change NGINX to use TCP? As mentioned before it does not work using http, if your output is Logstash, then you need to use TCP.

You need to configure Nginx to work as a SSL TCP Pass-through proxy.

marlonws · April 29, 2024, 7:29pm

I was trying to eliminate other errors, but it seems that in the end this is what it is

Rios · April 29, 2024, 7:49pm

Well, you have multiple points, need to go step by step.

HANDSHAKEN: protocol:TLSv1.2 cipher suite:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

TLS is not issue.

[2024-04-29T19:05:33,293][DEBUG][logstash.codecs.plain ][elastic-agent-pipeline][442d31ff9574bceceb399544e71c7bf443ffa611f38e8e52ab291cf580761ddd] config LogStash::Codecs::Plain/@charset = "UTF-8"

LS uses UTF-8 encoding by default.
The nginx log encoding issue is described here. Most likely you have to set UTF-8.

I couldn't get what now come to LS as an event/message, nothing or something with the wrong encoding.

Try what Leandro suggested, switch to TCP.

marlonws · April 30, 2024, 10:46pm

Back to the subject

I removed nginx and logstash worked fine. The problem really was via TCP.

To avoid wasting time, I left out nginx and placed the vm behind the application gateway with the initial tpc option to do the tests and it worked too, now it's time to configure tls/tpc. And the vm already has no public IP, it only responds to the IP of the application load balancer.

Logstash would never work within the cluster using https, that was the problem so haha, in the future I will try to put logstash within the cluster again.

marlonws · May 1, 2024, 7:28am

I have a question, the logstash port 9600 monitoring api, I need to use it as a health-probe, but as input 5044 is TCL/TLS, port 9600 needs to be the same.

Is the SSL of the 9600 that uses the keystore tcp/tls or https?