Logstash not listening port 5044 even when we configured the beats

Elastic Stack Logstash
1

input {
beats {
type => beats
host => "localhost"
port => 5044
}

2

it is always listens to the 9600 only

3

take out this line.. and try again.

# host => "localhost"

4

Still not working

5

You are going to need to provide more information if you want help.

What version are you on?

How do you know it is not working / how did you test?

What did the logstash startup logs show?

The logs should show somthing like this if it started succesfully.

2022-07-27T16:12:45,182][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.47}
[2022-07-27T16:12:45,204][INFO ][logstash.inputs.beats    ][main] Starting input listener {:address=>"0.0.0.0:5044"}
[2022-07-27T16:12:45,215][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}

And if not something like

[2022-07-27T16:15:22,216][INFO ][logstash.inputs.beats    ][main] Starting input listener {:address=>"0.0.0.0:5044"}
[2022-07-27T16:17:35,779][INFO ][org.logstash.beats.Server][main][cb9c0e56545925dd4243dfb98dfe230729cbf8579eaa7521ef7bb161b15019d2] Starting server on port: 5044
[2022-07-27T16:17:42,050][ERROR][logstash.javapipeline    ][main][cb9c0e56545925dd4243dfb98dfe230729cbf8579eaa7521ef7bb161b15019d2] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::Beats port=>5044, id=>"cb9c0e56545925dd4243dfb98dfe230729cbf8579eaa7521ef7bb161b15019d2", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_c29fc438-7de4-4977-9b65-37c420c1f308", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, add_hostname=>false, ssl_verify_mode=>"none", ssl_peer_metadata=>false, include_codec_tag=>true, ssl_handshake_timeout=>10000, tls_min_version=>1, tls_max_version=>1.3, cipher_suites=>["TLS_AES_128_GCM_SHA256", "TLS_AES_256_GCM_SHA384", "TLS_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], client_inactivity_timeout=>60, executor_threads=>16>
  Error: Address already in use
  Exception: Java::JavaNet::BindException
  Stack: sun.nio.ch.Net.bind0(Native Method)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:459)
.....
6

I tried to run in the docker container and exposed the ports 5044 .Logstash is running.But when I try to curl with 5044 it is not listening.using the netstat -plnt.

Please help me to check on this issue.Id possible please provide any link to connect with you to show what exactly the problem is.

Thanks,
Kala

7

Hi @KALAVATHI_YALAMANCHA

This is a community forum not personal/ commercial support...

This is the first time you've mentioned Docker

The best way to get help is to be complete in the information you're providing, not just little pieces of information.

Share your docker file / compose / docker command and the logs.

Did you read the docs?

Sounds to me like like your logstash conf file is not being read / used.

You still have not provided logs nor the commands you are starting logstash nor you configuration... We can't help until you do

1 Like
9

docker run -it imageid

10

Your issue has nothing to do with Logstash, it is a docker issue, you are not exposing the port, please check the docker documentation.

The EXPOSE instruction does not actually publish the port. It functions as a type of documentation between the person who builds the image and the person who runs the container, about which ports are intended to be published. To actually publish the port when running the container, use the -p flag on docker run to publish and map one or more ports, or the -P flag to publish all exposed ports and map them to high-order ports.

You need to use -p 5044:5044 in your docker run command.

1 Like
11

i dont run the docker command manually.I have created ecs task in aws and using this image the task will run.Is there any way to run automatically without providng the port manually?

12

I do not use docker much nor AWS so I can not help further, but your issue is that you are not publishing the port.

Just having EXPOSE port-number in your dockerfile will not publish the port, you need to tell docker which is the port you will use in the docker host and the internal port in the docker container.

How you do that depends entirely in the way you are starting docker, you could edit your task to pass the -p 5044:5044 parameter, you can use docker-compose and have the ports in the configuration and probably there are many other ways, it is better for you to check on a docker forum since your issue is not related to any Elastic products.

1 Like
13

curl: (56) Recv failure: Connection reset by peer when i ping the localhost with 5044 port

14

@KALAVATHI_YALAMANCHA I agree with @leandrojmp this is a docker network issue not exposing ports... your docker execution is incorrect and the 5044 is not actually expose on the host network.

The EXPOSE instruction does not actually publish the port. It functions as a type of documentation between the person who builds the image and the person who runs the container, about which ports are intended to be published. To actually publish the port when running the container, use the -p flag on docker run to publish and map one or more ports, or the -P flag to publish all exposed ports and map them to high-order ports.

Here is a sample complete docker-compose solution

hyperion:logstash sbrown$ pwd
/Users/sbrown/workspace/elastic-install/docker/logstash
hyperion:logstash sbrown$ ls
config/                 docker-compose.yml      pipeline/
hyperion:logstash sbrown$ 
hyperion:logstash sbrown$ cat docker-compose.yml 
version: "3.3"

services:
  logstash:
    image: docker.elastic.co/logstash/logstash:8.3.2
    ports:
      - 5044:5044
    volumes:
      - ~/workspace/elastic-install/docker/logstash/pipeline/:/usr/share/logstash/pipeline/:ro
      - ~/workspace/elastic-install/docker/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro
      - ~/workspace/elastic-install/docker/logstash/config/pipelines.yml:/usr/share/logstash/config/pipelines.yml:ro


hyperion:logstash sbrown$ 
hyperion:logstash sbrown$ cat config/logstash.yml 
http.host: "0.0.0.0"
config.reload.automatic: true

hyperion:logstash sbrown$ 
hyperion:logstash sbrown$ cat config/pipelines.yml 
- pipeline.id: main
  path.config: "/usr/share/logstash/pipeline"

hyperion:logstash sbrown$ 
hyperion:logstash sbrown$ cat pipeline/logstash-sample.conf 
# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => ["http://host.docker.internal:9200"]
    index => "%{[@metadata][beat]}-%{[@metadata][version]}"
    action => "create"
  }
  # stdout {
  #   codec => "rubydebug"
  # }
}
hyperion:logstash sbrown$

See my docker ps shows the mapping to between internal and external ports your only shows one side of the mapping thus the port is not expose to the host... it never will be with your current setup.

0.0.0.0:5044->5044/tcp

hyperion:docker sbrown$ docker ps | grep logstash
2b10a664b693   docker.elastic.co/logstash/logstash:8.3.2             "/usr/local/bin/dock…"   6 minutes ago   Up 6 minutes   0.0.0.0:5044->5044/tcp, 9600/tcp   logstash-logstash-1

hyperion:docker sbrown$ sudo lsof -i -P | grep LISTEN | grep :$PORT | grep sbrown
com.docke  4207         sbrown   63u  IPv6 0xfed9fc326875e059      0t0    TCP *:5601 (LISTEN)
com.docke  4207         sbrown  133u  IPv6 0xfed9fc3268760a59      0t0    TCP *:9200 (LISTEN)
com.docke  4207         sbrown  182u  IPv6 0xfed9fc3268757059      0t0    TCP *:5044 (LISTEN)
15

Thanks for your reply. I am not getting the output for the incoming beats.Can you please help to suggest on this.
i am sending winlog beats but i could nt able to find the beats in the logstash output.Please help to check on this?

i have configured the winlog beat output to send to logstash like below
output.logstash:

The Logstash hosts

hosts: ["x.x.x.x:5044","x.x.x.x:5044"]
loadbalance: true

i made the connection by using the test output
PS C:\Users\Administrator\Downloads\winlogbeat> .\winlogbeat.exe test output
logstash: x.x.x.x:5044...
connection...
parse host... OK
dns lookup... OK
addresses: x.x.x.x
dial up... OK
TLS... WARN secure connection disabled
talk to server... OK

16

So looks like perhaps you got the logstash port opened?

When you run winlogbeat does it show connected?

if youy logstash conf looks like

nput {
beats {
port => 5044
type => beats
}
}
output
{
stdout { }
}

then the output would be in the stdout of the docker container... no where else...

I would put

stdout { codec => "rubydebug"}

Keep working on it...

18

You will need to exec into the docker container to see the output you are just writing it to stdout inside the container...

Where do you actually want to send the results?

19

i want to see the output in the logs itself.

20

Thank you so much both of you.I got the desired output.For filtering can you suggest any format for windows logs?