# Logstash communication | configuration

**URL:** <https://discuss.elastic.co/t/logstash-communication-configuration/275331>\
**Category:** Logstash\
**Created:** [June 8, 2021, 5:49pm UTC](https://discuss.elastic.co/t/logstash-communication-configuration/275331 "2021-06-08T17:49:49Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![farciarz121](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@farciarz121](https://discuss.elastic.co/u/farciarz121)\
**Post date:** [June 8, 2021, 5:49pm UTC](https://discuss.elastic.co/t/logstash-communication-configuration/275331/1 "2021-06-08T17:49:49Z")

</div>

I am traying to send logs from beats to logs to ES cloud.

5 beats are hosted on IP addresses from 192.168.1.10-15. Logstash is installed on 192.168.1.100 node.

Here is example of part of packet beat conf file from one of my nodes

```auto
output.logstash:
      hosts: ["192.168.1.100:5044"] 
      pipline: geoip-info 

```

Here is how conf file is configured in logstash ( thanks @stephenb)

```auto
################################################
# beats->logstash->es default config.
################################################
input {
  beats {
    port => 5044
  }
}

output {
  if [@metadata][pipeline] {
    elasticsearch {
      cloud_auth => "elastic:password"
      cloud_id => "mycloud:dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRj......"

      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}"
      pipeline => "%{[@metadata][pipeline]}" 
    }
  } else {
    elasticsearch {
      cloud_auth => "elastic:password"
      cloud_id => "mycloud:dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRj......"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}"
    }
  }
}

```

And yml from logstash as follows:

```auto
# Settings file in YAML
#
# Settings can be specified either in hierarchical form, e.g.:
#
# pipeline:
# batch:
# size: 125
# delay: 5
#
# Or as flat keys:
#
# pipeline.batch.size: 125
# pipeline.batch.delay: 5
#
# ------------ Node identity ------------
#
# Use a descriptive name for the node:
#
node.name: logstash-node1
#
# If omitted the node name will default to the machine's host name
#
# ------------ Data path ------------------
#
# Which directory should be used by logstash and its plugins
# for any persistent needs. Defaults to LOGSTASH_HOME/data
#
# path.data:
#
# ------------ Pipeline Settings --------------
#
# The ID of the pipeline.
pipeline.id: main
#
# Set the number of workers that will, in parallel, execute the filters+outputs
# stage of the pipeline.
#
# This defaults to the number of the host's CPU cores.
#
pipeline.workers: 2
#
# How many events to retrieve from inputs before sending to filters+workers
#
# pipeline.batch.size: 125
#
# How long to wait in milliseconds while polling for the next event
# before dispatching an undersized batch to filters+outputs
#
# pipeline.batch.delay: 50
#
# Force Logstash to exit during shutdown even if there are still inflight
# events in memory. By default, logstash will refuse to quit until all
# received events have been pushed to the outputs.
#
# WARNING: enabling this can lead to data loss during shutdown
#
# pipeline.unsafe_shutdown: false
#
# Set the pipeline event ordering. Options are "auto" (the default), "true" or "false".
# "auto" will automatically enable ordering if the 'pipeline.workers' setting
# is also set to '1'.
# "true" will enforce ordering on the pipeline and prevent logstash from starting
# if there are multiple workers.
# "false" will disable any extra processing necessary for preserving ordering.
#
#pipeline.ordered: auto
#
# ------------ Pipeline Configuration Settings --------------
#
# Where to fetch the pipeline configuration for the main pipeline
#
#path.config: C:/logstash-7.12.1/config/logstash-sample.conf
#
# Pipeline configuration string for the main pipeline
#
# config.string:
#
# At startup, test if the configuration is valid and exit (dry run)
#
# config.test_and_exit: false
#
# Periodically check if the configuration has changed and reload the pipeline
# This can also be triggered manually through the SIGHUP signal
#
# config.reload.automatic: false
#
# How often to check if the pipeline configuration has changed (in seconds)
# Note that the unit value (s) is required. Values without a qualifier (e.g. 60) 
# are treated as nanoseconds.
# Setting the interval this way is not recommended and might change in later versions.
#
# config.reload.interval: 3s
#
# Show fully compiled configuration as debug log message
# NOTE: --log.level must be 'debug'
#
#config.debug: true
#
# When enabled, process escaped characters such as \n and \" in strings in the
# pipeline configuration files.
#
# config.support_escapes: false
#

# on it will not work as intended.
# http.enabled: true
#
# By default, the HTTP API is bound to only the host's local loopback interface,
# ensuring that it is not accessible to the rest of the network. Because the API
# includes neither authentication nor authorization and has not been hardened or
# tested for use as a publicly-reachable API, binding to publicly accessible IPs
# should be avoided where possible.
#
# http.host: 127.0.0.1
#
# The HTTP API web server will listen on an available port from the given range.
# Values can be specified as a single port (e.g., `9600`), or an inclusive range
# of ports (e.g., `9600-9700`).
#
# http.port: 9600-9700

# ------------ Cloud Settings ---------------
# Define Elastic Cloud settings here.
# Format of cloud.id is a base64 value e.g. dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRub3RhcmVhbCRpZGVudGlmaWVy
# and it may have an label prefix e.g. staging:dXMtZ...
# This will overwrite 'var.elasticsearch.hosts' and 'var.kibana.host'
#cloud.id: "security-deployment:ZWFzdHVzMi5henVyZS5lbGFzdGljLWNsb3VkLmNvbTo5MjQzJDI0ZThkODgwYjVhZDQ4Y2FhMzkxNjA4NjU3YjQ2ODNmJDIyMTgxYWIzMDNhMTQ4NDBhNDhmODllYWU5MjU4MDcz"

#cloud.auth: "elastic:yXrLSSfWDPcmHYhCNK0Kx8RQ"
# Format of cloud.auth is: <user>:<pass>
# This is optional
# If supplied this will overwrite 'var.elasticsearch.username' and 'var.elasticsearch.password'
# If supplied this will overwrite 'var.kibana.username' and 'var.kibana.password'
#cloud.auth: elastic:yXrLSSfWDPcmHYhCNK0Kx8RQ
#
# ------------ Queuing Settings --------------
#
# Internal queuing model, "memory" for legacy in-memory based queuing and
# "persisted" for disk-based acked queueing. Defaults is memory
#
# queue.type: memory
#
# If using queue.type: persisted, the directory path where the data files will be stored.
# Default is path.data/queue
#
# path.queue:
#
# If using queue.type: persisted, the page data files size. The queue data consists of
# append-only data files separated into pages. Default is 64mb
#
# queue.page_capacity: 64mb
#
# If using queue.type: persisted, the maximum number of unread events in the queue.
# Default is 0 (unlimited)
#
# queue.max_events: 0
#
# If using queue.type: persisted, the total capacity of the queue in number of bytes.
# If you would like more unacked events to be buffered in Logstash, you can increase the
# capacity using this setting. Please make sure your disk drive has capacity greater than
# the size specified here. If both max_bytes and max_events are specified, Logstash will pick
# whichever criteria is reached first
# Default is 1024mb or 1gb
#
# queue.max_bytes: 1024mb
#
# If using queue.type: persisted, the maximum number of acked events before forcing a checkpoint
# Default is 1024, 0 for unlimited
#
# queue.checkpoint.acks: 1024
#
# If using queue.type: persisted, the maximum number of written events before forcing a checkpoint
# Default is 1024, 0 for unlimited
#
# queue.checkpoint.writes: 1024
#
# If using queue.type: persisted, the interval in milliseconds when a checkpoint is forced on the head page
# Default is 1000, 0 for no periodic checkpoint.
#
# queue.checkpoint.interval: 1000
#
# ------------ Dead-Letter Queue Settings --------------
# Flag to turn on dead-letter queue.
#
# dead_letter_queue.enable: false

# If using dead_letter_queue.enable: true, the maximum size of each dead letter queue. Entries
# will be dropped if they would increase the size of the dead letter queue beyond this setting.
# Default is 1024mb
# dead_letter_queue.max_bytes: 1024mb

# If using dead_letter_queue.enable: true, the interval in milliseconds where if no further events eligible for the DLQ
# have been created, a dead letter queue file will be written. A low value here will mean that more, smaller, queue files
# may be written, while a larger value will introduce more latency between items being "written" to the dead letter queue, and
# being available to be read by the dead_letter_queue input when items are are written infrequently.
# Default is 5000.
#
# dead_letter_queue.flush_interval: 5000

# If using dead_letter_queue.enable: true, the directory path where the data files will be stored.
# Default is path.data/dead_letter_queue
#
# path.dead_letter_queue:
#
# ------------ Metrics Settings --------------
#
# Bind address for the metrics REST endpoint
#
# http.host: "127.0.0.1"
#
# Bind port for the metrics REST endpoint, this option also accept a range
# (9600-9700) and logstash will pick up the first available ports.
#
# http.port: 9600-9700
#
# ------------ Debugging Settings --------------
#
# Options for log.level:
# * fatal
# * error
# * warn
# * info (default)
# * debug
# * trace
#
# log.level: info
# path.logs:
#
# ------------ Other Settings --------------
#
# Where to find custom plugins
# path.plugins: []
#
# Flag to output log lines of each pipeline in its separate log file. Each log filename contains the pipeline.name
# Default is false
# pipeline.separate_logs: false
#
# ------------ X-Pack Settings (not applicable for OSS build)--------------
#
# X-Pack Monitoring
# https://www.elastic.co/guide/en/logstash/current/monitoring-logstash.html
#xpack.monitoring.enabled: false
#xpack.monitoring.elasticsearch.username: logstash_admin
#xpack.monitoring.elasticsearch.password: Imported88!
#xpack.monitoring.elasticsearch.proxy: ["http://proxy:port"]
#xpack.monitoring.elasticsearch.hosts: ["https://es1:9200", "https://es2:9200"]
# an alternative to hosts + username/password settings is to use cloud_id/cloud_auth
#xpack.monitoring.elasticsearch.cloud_id: ZWFzdHVzMi5henVyZS5lbGFzdGljLWNsb3VkLmNvbTo5MjQzJDI0ZThkODgwYjVhZDQ4Y2FhMzkxNjA4NjU3YjQ2ODNmJDIyMTgxYWIzMDNhMTQ4NDBhNDhmODllYWU5MjU4MDcz
#xpack.monitoring.elasticsearch.cloud_auth: elastic:yXrLSSfWDPcmHYhCNK0Kx8RQ
# another authentication alternative is to use an Elasticsearch API key
#xpack.monitoring.elasticsearch.api_key: "id:api_key"
#xpack.monitoring.elasticsearch.ssl.certificate_authority: ["/path/to/ca.crt"]
#xpack.monitoring.elasticsearch.ssl.truststore.path: path/to/file
#xpack.monitoring.elasticsearch.ssl.truststore.password: password
#xpack.monitoring.elasticsearch.ssl.keystore.path: /path/to/file
#xpack.monitoring.elasticsearch.ssl.keystore.password: password
#xpack.monitoring.elasticsearch.ssl.verification_mode: certificate
#xpack.monitoring.elasticsearch.sniffing: false
#xpack.monitoring.collection.interval: 10s
#xpack.monitoring.collection.pipeline.details.enabled: true
#
# X-Pack Management
# https://www.elastic.co/guide/en/logstash/current/logstash-centralized-pipeline-management.html
xpack.management.enabled: false
#xpack.management.pipeline.id: ["main", "apache_logs"]
#xpack.management.elasticsearch.username: logstash_xxxx
#xpack.management.elasticsearch.password: xxxxxx
#xpack.management.elasticsearch.proxy: ["http://proxy:port"]
#xpack.management.elasticsearch.hosts: ["https://es1:9200", "https://es2:9200"]
# an alternative to hosts + username/password settings is to use cloud_id/cloud_auth
xpack.management.elasticsearch.cloud_id: xxxx-deployment:ZxxxxxxxFhMzkxNjA4NjU3YjQ2ODNmJDIyMTgxYWIzMDNhMTQ4NxxxxU5MjU4MDcz
xpack.management.elasticsearch.cloud_auth: elastic:ks9LIaWkVHpzxUJ3rYvfjIer
# another authentication alternative is to use an Elasticsearch API key
#xpack.management.elasticsearch.api_key: "id:api_key"
#xpack.management.elasticsearch.ssl.certificate_authority: ["/path/to/ca.crt"]
#xpack.management.elasticsearch.ssl.truststore.path: /path/to/file
#xpack.management.elasticsearch.ssl.truststore.password: password
#xpack.management.elasticsearch.ssl.keystore.path: /path/to/file
#xpack.management.elasticsearch.ssl.keystore.password: password
#xpack.management.elasticsearch.ssl.verification_mode: certificate
#xpack.management.elasticsearch.sniffing: false
#xpack.management.logstash.poll_interval: 5s

```

I have created firewall rules on logstash and I know beat can communicate with logstash, however logstash is not forwarding data correctly to Elastic cloud because I can not see new data coming in Kibana. Any suggestions where problem can be? Is logstash listening on default from all ip addresses?

PS. Everything works fine if I setup beat to send logs directly to ES cloud. The problem starts if I direct logs to logstash and from logstash to ES cloud. Also, if I send logs from beat installed locally on logstash everything works fine.

```auto
# By default, the HTTP API is bound to only the host's local loopback interface,
# ensuring that it is not accessible to the rest of the network 

```

The solution is probably here, but how do I define what addresses logstash should listen from

---

<div class="post-metadata">

**Author:** ![farciarz121](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@farciarz121](https://discuss.elastic.co/u/farciarz121)\
**Post date:** [June 9, 2021, 12:41pm UTC](https://discuss.elastic.co/t/logstash-communication-configuration/275331/2 "2021-06-09T12:41:57Z")

</div>

please advise

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 9, 2021, 1:16pm UTC](https://discuss.elastic.co/t/logstash-communication-configuration/275331/3 "2021-06-09T13:16:29Z")

</div>

@farciarz121 Please be patient as only volunteers staff this forum.

> [@farciarz121](#):
>
> ```auto
> input {
> beats {
> port => 5044
> }
> }
> 
> ```

The above code defines which port logstash will listen on for beats it will be listening on port 5044.

Which is were you are pointing beats to with this

```auto
output.logstash:
      hosts: ["192.168.1.100:5044"] 
      pipline: geoip-info 

```

Steps to debug.

1. Try to telnet from a filebeat host to the logstash host on port 5044  
From filebeat host it should connect.  
`telnet <logstaship> 5044`

2. from Logstah host try to curl elastic cloud. You can get the elasticsearch URL from the cloud console, make sure you

`curl -u username:password https://elasticsearchurl`

1. Note logstash monitoring port runs on localhost:9600 but unless that port is already consumed you don't need to worry about that. You can check that with

`curl -u username:password https://elasticsearchurl`

1. Important, always Provide the startup logs from logstash which you should always do when asking a question. The logs will mostly likely have the issue in them.

---

<div class="post-metadata">

**Author:** ![farciarz121](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@farciarz121](https://discuss.elastic.co/u/farciarz121)\
**Post date:** [June 9, 2021, 1:40pm UTC](https://discuss.elastic.co/t/logstash-communication-configuration/275331/4 "2021-06-09T13:40:10Z")

</div>

Hi @stephenb , my apologies. I do appreciate help and every single input from all forum members. I am really impressed with the elastic community. It sometimes happens that a new topic gets berried under other topics and forums members do not have a chance to see it.

Regarding to your suggestions, I will try these steps and will get back to you. Also, I will remember to post startup logs next time. Thank you

---

<div class="post-metadata">

**Author:** ![farciarz121](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@farciarz121](https://discuss.elastic.co/u/farciarz121)\
**Post date:** [June 10, 2021, 6:45pm UTC](https://discuss.elastic.co/t/logstash-communication-configuration/275331/5 "2021-06-10T18:45:26Z")

</div>

> [@stephenb](#):
>
> Important, always Provide the startup logs from logstash which you should always do when asking a question. The logs will mostly likely have the issue in them.

```auto
PS C:\logstash-7.12.1-windows-x86_64\logstash-7.12.1> .\bin\logstash.bat -f .\config\logstash.conf
"Using bundled JDK: ""
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in
a future release.
Sending Logstash logs to C:/logstash-7.12.1-windows-x86_64/logstash-7.12.1/logs which is now configured via log4j2.prope
rties
[2021-06-10T15:45:38,199][INFO][logstash.runner] Log4j configuration path used is: C:\logstash-7.12.1-windows
-x86_64\logstash-7.12.1\config\log4j2.properties
[2021-06-10T15:45:38,209][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.12.1", "jruby.vers
ion"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 11.0.10+9 on 11.0.10+9 +indy +jit [mswin32-
x86_64]"}
[2021-06-10T15:45:38,313][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or
 command line options are specified
[2021-06-10T15:45:39,238][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2021-06-10T15:45:40,808][INFO][org.reflections.Reflections] Reflections took 41 ms to scan 1 urls, producing 23 keys a
nd 47 values
[2021-06-10T15:45:41,815][INFO][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:remo
ved=>[], :added=>[https://elastic:xxxxxx@24e8d880b5ad48caa391608657b4683f.eastus2.azure.elastic-cloud.com:9243/]}}
[2021-06-10T15:45:42,284][WARN][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"https:
//elastic:xxxxxx@24e8d880b5ad48caa391608657b4683f.eastus2.azure.elastic-cloud.com:9243/"}
[2021-06-10T15:45:42,443][INFO][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
[2021-06-10T15:45:42,446][WARN][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` even
t field won't be used to determine the document _type {:es_version=>7}
[2021-06-10T15:45:42,507][INFO][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outp
uts::ElasticSearch", :hosts=>["https://2xxxx.eastus2.azure.elastic-cloud.com:9243"]}
[2021-06-10T15:45:42,538][INFO][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:remo
ved=>[], :added=>[https://elastic:xxxxxx@24xxxxxb4683f.eastus2.azure.elastic-cloud.com:9243/]}}
[2021-06-10T15:45:42,659][WARN][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"https:
//elastic:xxxxxx@24e8d880b5adxxxxxxxb4683f.eastus2.azure.elastic-cloud.com:9243/"}
[2021-06-10T15:45:42,783][INFO][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
[2021-06-10T15:45:42,786][WARN][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` even
t field won't be used to determine the document _type {:es_version=>7}
[2021-06-10T15:45:42,834][INFO][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outp
uts::ElasticSearch", :hosts=>["https://24e8xxxxxx57b4683f.eastus2.azure.elastic-cloud.com:9243"]}
[2021-06-10T15:45:42,925][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline_id=>"main", "pipeline.wor
kers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, "pipeline.sources"=>["C:
/logstash-7.12.1-windows-x86_64/logstash-7.12.1/config/logstash.conf"], :thread=>"#<Thread:0x2bafa0de run>"}
[2021-06-10T15:45:43,766][INFO][logstash.javapipeline][main] Pipeline Java execution initialization time {"seconds"
=>0.84}
[2021-06-10T15:45:43,816][INFO][logstash.inputs.beats][main] Starting input listener {:address=>"0.0.0.0:5044"}
[2021-06-10T15:45:43,834][INFO][logstash.javapipeline][main] Pipeline started {"pipeline.id"=>"main"}
[2021-06-10T15:45:43,914][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :
non_running_pipelines=>[]}
[2021-06-10T15:45:43,960][INFO][org.logstash.beats.Server][main][e5980c2f89ab7ab6e256ddce0d1310850166358d3ea7245cf6df7b
5e802f2d40] Starting server on port: 5044

```

Still not sucessfull. I have completely turned off Windows firewall on both machines.

I am trying to send logs from 192.168.1.205 to 192.158.1.150 (logstash).  
Pfsense rules as follow:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/2/82de38930332e6450d4cbe575706f4d3885f5caf.png)

And here is what I get when I try to connect with telnet

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/7/17de72677a1a3e15611f1fdf87d6ba4099365d41.png)

Just black screen...

I'll also add that logstash is install on Win Server 2016

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 10, 2021, 6:58pm UTC](https://discuss.elastic.co/t/logstash-communication-configuration/275331/6 "2021-06-10T18:58:00Z")

</div>

What is the IP of your machines?

You said that your logstash is using the IP `192.158.1.150`, but I will assume this is a type and the correct IP is `192.168.1.150`, which is the IP in your telnet screen and your screenshot.

But your packet beat configuration is pointing to another IP address, which is `192.168.1.150`.

```auto
output.logstash:
      hosts: ["192.168.1.100:5044"] 
      pipline: geoip-info 

```

What is the correct IP address? Check your configuration to see if you are using the correct IP address.

The Telnet blank screen means that the connection was established.

---

<div class="post-metadata">

**Author:** ![farciarz121](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@farciarz121](https://discuss.elastic.co/u/farciarz121)\
**Post date:** [June 10, 2021, 7:08pm UTC](https://discuss.elastic.co/t/logstash-communication-configuration/275331/7 "2021-06-10T19:08:14Z")

</div>

Oh gosh... That was a silly mistake... Sorry, in the process of learning, I have reconfigured it so many times that I completely overlooked this. You are right, configuration file was pointing to the wrong address. I have updated my configuration file and life is good again...

Well maybe not 100% ... good  
still getting some errors

```auto
[2021-06-10T15:45:41,815][INFO][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:remo
ved=>[], :added=>[https://elastic:xxxxxx@24e8d880b5ad48caa391608657b4683f.eastus2.azure.elastic-cloud.com:9243/]}}
[2021-06-10T15:45:42,284][WARN][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"https:
//elastic:xxxxxx@24e8d880b5ad48caa391608657b4683f.eastus2.azure.elastic-cloud.com:9243/"}
[2021-06-10T15:45:42,443][INFO][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
[2021-06-10T15:45:42,446][WARN][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` even
t field won't be used to determine the document _type {:es_version=>7}
[2021-06-10T15:45:42,507][INFO][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outp
uts::ElasticSearch", :hosts=>["https://24e8d880b5ad48caa391608657b4683f.eastus2.azure.elastic-cloud.com:9243"]}
[2021-06-10T15:45:42,538][INFO][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:remo
ved=>[], :added=>[https://elastic:xxxxxx@24e8d880b5ad48caa391608657b4683f.eastus2.azure.elastic-cloud.com:9243/]}}
[2021-06-10T15:45:42,659][WARN][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"https:
//elastic:xxxxxx@24e8d880b5ad48caa391608657b4683f.eastus2.azure.elastic-cloud.com:9243/"}
[2021-06-10T15:45:42,783][INFO][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
[2021-06-10T15:45:42,786][WARN][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` even
t field won't be used to determine the document _type {:es_version=>7}
[2021-06-10T15:45:42,834][INFO][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outp
uts::ElasticSearch", :hosts=>["https://24e8d8xxxx7b4683f.eastus2.azure.elastic-cloud.com:9243"]}
[2021-06-10T15:45:42,925][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline_id=>"main", "pipeline.wor
kers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, "pipeline.sources"=>["C:
/logstash-7.12.1-windows-x86_64/logstash-7.12.1/config/logstash.conf"], :thread=>"#<Thread:0x2bafa0de run>"}
[2021-06-10T15:45:43,766][INFO][logstash.javapipeline][main] Pipeline Java execution initialization time {"seconds"
=>0.84}
[2021-06-10T15:45:43,816][INFO][logstash.inputs.beats][main] Starting input listener {:address=>"0.0.0.0:5044"}
[2021-06-10T15:45:43,834][INFO][logstash.javapipeline][main] Pipeline started {"pipeline.id"=>"main"}
[2021-06-10T15:45:43,914][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :
non_running_pipelines=>[]}
[2021-06-10T15:45:43,960][INFO][org.logstash.beats.Server][main][e5980c2f89ab7ab6e256ddce0d1310850166358d3ea7245cf6df7b
5e802f2d40] Starting server on port: 5044
[2021-06-10T15:53:54,597][INFO][org.logstash.beats.BeatsHandler][main][e5980c2f89ab7ab6e256ddce0d1310850166358d3ea7245c
f6df7b5e802f2d40] [local: 192.168.1.150:5044, remote: 192.168.1.205:63768] Handling exception: io.netty.handler.codec.De
coderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 92 (caused by: org.
logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 92)
[2021-06-10T15:53:54,601][WARN][io.netty.channel.DefaultChannelPipeline][main][e5980c2f89ab7ab6e256ddce0d1310850166358d
3ea7245cf6df7b5e802f2d40] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually m
eans the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats prot
ocol: 92
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:471) ~[netty-all-4.1.49.Fina
l.jar:4.1.49.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-all-4.1.49.Fin
al.jar:4.1.49.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[ne
tty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61) ~[netty-all-
4.1.49.Final.jar:4.1.49.Final]
        at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370) ~[netty-all-4.1.
49.Final.jar:4.1.49.Final]
        at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.49.Final.jar:
4.1.49.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-all-4.1.4
9.Final.jar:4.1.49.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.49.Final.jar:4.1.49.
Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.49.Final
.jar:4.1.49.Final]
        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 92
        at org.logstash.beats.Protocol.version(Protocol.java:22) ~[logstash-input-beats-6.1.2.jar:?]
        at org.logstash.beats.BeatsParser.decode(BeatsParser.java:62) ~[logstash-input-beats-6.1.2.jar:?]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) ~[n
etty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) ~[netty-all-4.1.49.Fina
l.jar:4.1.49.Final]

```

```auto
PS C:\Windows\system32> cd "C:\Program Files (x86)\winlogbeat"
PS C:\Program Files (x86)\winlogbeat> .\winlogbeat.exe -e
2021-06-10T15:06:47.196-0800 INFO instance/beat.go:665 Home path: [C:\Program Files (x86)\winlogbeat] Config path: [C:\Program Files (x86)\winlogbeat] Data path: [C:\Program Files (x86)\winlogbeat\data] Logs path: [C:\Program Files (x86)\winlogbeat\logs]
2021-06-10T15:06:47.197-0800 INFO instance/beat.go:673 Beat ID: 3d395a55-2444-417d-ad73-c819727ad837
2021-06-10T15:06:47.213-0800 INFO [beat] instance/beat.go:1014 Beat info {"system_info": {"beat": {"path": {"config": "C:\\Program Files (x86)\\winlogbeat", "data": "C:\\Program Files (x86)\\winlogbeat\\data", "home": "C:\\Program Files (x86)\\winlogbeat", "logs": "C:\\Program Files (x86)\\winlogbeat\\logs"}, "type": "winlogbeat", "uuid": "3d395a55-2444-417d-ad73-c819727ad837"}}}
2021-06-10T15:06:47.213-0800 INFO [beat] instance/beat.go:1023 Build info {"system_info": {"build": {"commit": "054e224d226b42a1dd7c72dcf48c3f18de452e22", "libbeat": "7.13.0", "time": "2021-05-19T22:47:56.000Z", "version": "7.13.0"}}}
2021-06-10T15:06:47.213-0800 INFO [beat] instance/beat.go:1026 Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":1,"version":"go1.15.12"}}}
2021-06-10T15:06:47.227-0800 INFO [beat] instance/beat.go:1030 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2021-06-10T13:57:54.95-08:00","name":"DESKTOP_1","ip":["fe80::4d0f:4596:ccf9:e17e/64","192.168.1.205/24","::1/128","127.0.0.1/8"],"kernel_version":"10.0.19041.985 (WinBuild.160101.0800)","mac":["00:0c:29:0e:4b:84"],"os":{"type":"windows","family":"windows","platform":"windows","name":"Windows 10 Pro","version":"10.0","major":10,"minor":0,"patch":0,"build":"19042.985"},"timezone":"AKDT","timezone_offset_sec":-28800,"id":"dc186c98-07cb-471f-a5c7-fefad9d6cecd"}}}
2021-06-10T15:06:47.230-0800 INFO [beat] instance/beat.go:1059 Process info {"system_info": {"process": {"cwd": "C:\\Program Files (x86)\\winlogbeat", "exe": "C:\\Program Files (x86)\\winlogbeat\\winlogbeat.exe", "name": "winlogbeat.exe", "pid": 1436, "ppid": 4340, "start_time": "2021-06-10T15:06:47.083-0800"}}}
2021-06-10T15:06:47.230-0800 INFO instance/beat.go:309 Setup Beat: winlogbeat; Version: 7.13.0
2021-06-10T15:06:47.230-0800 INFO [publisher] pipeline/module.go:113 Beat name: DESKTOP_1
2021-06-10T15:06:47.230-0800 INFO beater/winlogbeat.go:69 State will be read from and persisted to C:\Program Files (x86)\winlogbeat\data\.winlogbeat.yml
2021-06-10T15:06:47.282-0800 WARN [cfgwarn] registered_domain/registered_domain.go:61 BETA: The registered_domain processor is beta.
2021-06-10T15:06:47.351-0800 WARN [cfgwarn] registered_domain/registered_domain.go:61 BETA: The registered_domain processor is beta.
2021-06-10T15:06:47.377-0800 INFO instance/beat.go:473 winlogbeat start running.
2021-06-10T15:06:47.397-0800 INFO [monitoring] log/log.go:117 Starting metrics logging every 30s
2021-06-10T15:06:47.442-0800 WARN beater/eventlogger.go:124 EventLog[Microsoft-Windows-Sysmon/Operational] Open() error. No events will be read from this source. The specified channel could not be found.
2021-06-10T15:06:50.237-0800 INFO [add_cloud_metadata] add_cloud_metadata/add_cloud_metadata.go:101 add_cloud_metadata: hosting provider type not detected.
2021-06-10T15:06:51.244-0800 INFO [publisher_pipeline_output] pipeline/output.go:143 Connecting to backoff(async(tcp://192.168.1.150:5044))
2021-06-10T15:06:51.245-0800 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
2021-06-10T15:06:51.262-0800 INFO [publisher] pipeline/retry.go:223 done
2021-06-10T15:06:51.263-0800 INFO [publisher_pipeline_output] pipeline/output.go:151 Connection to backoff(async(tcp://192.168.1.150:5044)) established
2021-06-10T15:06:53.497-0800 INFO beater/eventlogger.go:88 EventLog[Security] successfully published 570 events
2021-06-10T15:06:53.500-0800 INFO beater/eventlogger.go:88 EventLog[Application] successfully published 344 events
2021-06-10T15:06:54.351-0800 INFO beater/eventlogger.go:88 EventLog[Security] successfully published 630 events
2021-06-10T15:06:55.153-0800 INFO beater/eventlogger.go:88 EventLog[Security] successfully published 700 events
2021-06-10T15:06:56.037-0800 INFO beater/eventlogger.go:88 EventLog[Security] successfully published 700 events
2021-06-10T15:06:57.019-0800 INFO beater/eventlogger.go:88 EventLog[Security] successfully published 700 events
2021-06-10T15:06:57.787-0800 INFO beater/eventlogger.go:88 EventLog[Security] successfully published 700 events
2021-06-10T15:06:58.632-0800 INFO beater/eventlogger.go:88 EventLog[Security] successfully published 700 events
2021-06-10T15:06:59.980-0800 INFO beater/eventlogger.go:88 EventLog[Security] successfully published 700 events
2021-06-10T15:07:01.415-0800 INFO beater/eventlogger.go:88 EventLog[Security] successfully published 700 events
2021-06-10T15:07:01.955-0800 INFO beater/eventlogger.go:88 EventLog[Security] successfully published 385 events
2021-06-10T15:07:17.407-0800 INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":781,"time":{"ms":781}},"total":{"ticks":7796,"time":{"ms":7796},"value":7796},"user":{"ticks":7015,"time":{"ms":7015}}},"handles":{"open":223},"info":{"ephemeral_id":"412bab9e-6e4e-4044-8681-1de8af2feba9","uptime":{"ms":30261}},"memstats":{"gc_next":42497360,"memory_alloc":38623896,"memory_sys":76498040,"memory_total":475005968,"rss":85790720},"runtime":{"goroutines":35}},"libbeat":{"config":{"module":

```

(please notice this is from winlog not packet beat, I decided it will be easier to set up winlog beat first)

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 10, 2021, 8:02pm UTC](https://discuss.elastic.co/t/logstash-communication-configuration/275331/8 "2021-06-10T20:02:57Z")

</div>

For the InvalidFrameProtocolException see [here](https://discuss.elastic.co/t/timestamp-not-match-with-the-log-date/262272/4).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 8, 2021, 8:03pm UTC](https://discuss.elastic.co/t/logstash-communication-configuration/275331/9 "2021-07-08T20:03:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.