# Syslog and filebeat assistance request **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [July 30, 2019, 9:14pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993 "2019-07-30T21:14:24Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [July 30, 2019, 9:14pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/1 "2019-07-30T21:14:24Z")
We have been running ELK v. 6.2.x We have a central syslog server on EL5 pushing log files with filebeat to logstash I have been tasked with setting up a new central syslog server on EL7 (replace EL5 instance). Filebeat-6.2.2 is the current version on the EL5 host, matched filebeat version on EL7 host. when testing the output its all ok. parse host... OK dns lookup... OK addresses.... 10.10.3.19 dial up... OK TLS... WARN talk to server... OK What I am not getting are the latest syslog details displaying in Kibana. Please tell me how I can verify the syslog data is getting to Logstash? Elasticsearch? Any advice would be much appreciated. TYIA! Please let me know if there are details you want to know about the environment. Sorry, I am an elk-noob. Brian ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 26, 2019, 3:21pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/2 "2019-08-26T15:21:44Z")
My /etc/filebeat/filebeat.yml . -- without comments `filebeat.inputs:` `- type: log` `enabled: true` `paths:` `- /var/log/messages` `- /var/log/asa` `- /var/log/asa-nat` `- /var/log/audit` `- /var/log/boot.log` `- /var/log/cron` `- /var/log/bsp-nas` `- /var/log/cups` `- /var/log/eduroam` `- /var/log/faillog` `- /var/log/lastlog` `- /var/log/mail` `- /var/log/maillog` `- /var/log/mcelog` `- /var/log/sa` `- /var/log/samba` `- /var/log/spooler` `- /var/log/tallylog` `- /var/log/up2date` `- /var/log/yum.log` `- /var/log/dhcpd-wifi-radius` `- /var/log/secure` `filebeat.config.modules:` `path: ${path.config}/modules.d/*.yml` `reload.enabled: false` `setup.template.settings:` `index.number_of_shards: 3` `setup.template.overwrite: true` `setup.kibana:` `output.logstash:` `hosts: ["logstash-host.mydomain.com:5044"]` ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 26, 2019, 7:39pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/3 "2019-08-26T19:39:37Z")
Getting this in the filebeat log on host 10.1.38.21 `2019-08-26T15:22:49.446-0500 ERROR logstash/async.go:256 Failed to publish events caused by: write tcp 10.1.38.21:53346->10.1.38.19:5044: write: connection reset by peer` `2019-08-26T15:22:51.007-0500 ERROR pipeline/output.go:121 Failed to publish events: write tcp 10.1.38.21:53346->10.1.38.19:5044: write: connection reset by peer` 10.1.38.21 is my filebeat server. 10.1.38.19 is my logstash server. ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 26, 2019, 8:25pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/4 "2019-08-26T20:25:22Z")
Is there any person on this forum that might provide some insight? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [August 27, 2019, 12:34am UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/5 "2019-08-27T00:34:44Z")
What does your logstash conf look like? And are you sure logstash is running and listening? Are you sure there is connectivity between the filebeat and logstash host? You should be able to test it by getting on the filebeat host and running `telnet 10.1.38.19 5044` It should show connection established if it can reach logstash on port 5044 ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 27, 2019, 1:29pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/6 "2019-08-27T13:29:27Z")
Thank you for your time! Yes I get a connection established when I telnet. when I run the `filebeat test output` it shows OK for connection. There is connection established in the logs, but when it tries to publish events the connection is reset by peer. `2019-08-26T16:37:10.167-0500 INFO pipeline/output.go:105 Connection to backoff(async(tcp://slp-b3c.its.bethel.edu:5044)) established` Logstash conf looks like this logstash.yml `---` `path.data: "/var/lib/logstash"` `path.logs: "/var/log/logstash"` `pipeline.batch.size: 125` `pipeline.batch.delay: 50` `config.reload.automatic: true` `config.reload.interval: 3s` `queue.type: memory` ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 27, 2019, 2:54pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/7 "2019-08-27T14:54:30Z")
Today I see a slightly different fail message `2019-08-27T09:53:25.316-0500 ERROR logstash/async.go:256 Failed to publish events caused by: read tcp 10.1.38.21:47796->10.1.38.19:3514: i/o timeout` ---
**Author:** ![justkind](https://avatars.discourse-cdn.com/v4/letter/j/919ad9/32.png) [@justkind](https://discuss.elastic.co/u/justkind)\ **Post date:** [August 27, 2019, 3:25pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/8 "2019-08-27T15:25:03Z")
Hey Brian, Can you share your logstash beats input configuration? Anything in the logs for logstash? The errors indicate an issue between filebeat and logstash so it could be an issue with the connectivity from filebeat to logstash (your logs show successful connection), the output plugin from filebeat (looks fine to me and logs show connection established), or the input plugin from logstash. The successful connection from telnet shows that logstash is up and listening on that port, but not sure if you have more advanced configuration on the input plugin. ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 27, 2019, 3:53pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/9 "2019-08-27T15:53:48Z")
here is my syslog.yml contents. `input {` `udp {` `type => 'syslog'` `port => 3514` `id => 'data1_udp'` `}` `tcp {` `type => 'syslog'` `port => 3514` `id => 'data1_tcp'` `}` `}` `filter {` `}` `output {` `#stdout { codec => rubydebug }` `kafka {` `codec => json` `topic_id => syslog` `bootstrap_servers => 'slp-3zp.its.bethel.edu:9092,slp-b3c.its.bethel.edu:9092'` `}` `}` Sorry about the formatting.. ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 27, 2019, 4:01pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/10 "2019-08-27T16:01:15Z")
Log on logstash not showing any matching times for the ERROR getting on filebeat. ---
**Author:** ![justkind](https://avatars.discourse-cdn.com/v4/letter/j/919ad9/32.png) [@justkind](https://discuss.elastic.co/u/justkind)\ **Post date:** [August 27, 2019, 4:25pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/11 "2019-08-27T16:25:59Z")
Thanks for the config info Brian, I understand your setup better now. Did you say this configuration was working previously? I'm not an elastic team member, just a dude who setup ELK at my workplace, so apologies if I'm wrong but as far as I understand it, you want your output to match up with your input plugin from logstash. In your filebeat configuration, you are sending log events to logstash on port 5044 which is the default port for the filebeat input plugin (for logstash): [https://www.elastic.co/guide/en/logstash/6.2/plugins-inputs-beats.html](https://www.elastic.co/guide/en/logstash/6.2/plugins-inputs-beats.html) You should have a beats input configured on logstash just like the link above if you want to receive events from filebeat on that port. More info in this link: [https://www.elastic.co/guide/en/beats/filebeat/6.2/logstash-output.html](https://www.elastic.co/guide/en/beats/filebeat/6.2/logstash-output.html) If you follow the link above, setup a beats input on your logstash and keep your filebeat configuration the same, you should definitely be able to see events coming into logstash. The current inputs you are using are UDP and TCP input plugins and would not affect your filebeat log events at all. Could be part of an old setup? If you get syslog configured on your source log boxes, and configure it to send to your logstash server on that specific port, 3514, you should then see your syslog logs but I have no experience setting up syslog so I can't confirm this. ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 27, 2019, 4:31pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/12 "2019-08-27T16:31:16Z")
Thanks for your help justkind! We do have an existing syslog server sending with filebeats - I noticed the ports did not match up, so I will dig deeper on that. Certainly stood out as not correct. ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 27, 2019, 4:32pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/13 "2019-08-27T16:32:36Z")
Not sure if this helps, here is the startup log from logstash. [2019-08-27T11:24:14,971][INFO][org.apache.kafka.clients.producer.ProducerConfig] ProducerConfig values: acks = 1 batch.size = 16384 bootstrap.servers = [[slp-3zp.mydomain.com:9092](http://slp-3zp.mydomain.com:9092), [slp-b3c.mydomain.com:9092](http://slp-b3c.mydomain.com:9092)] buffer.memory = 33554432 client.id = compression.type = none connections.max.idle.ms = 540000 enable.idempotence = false interceptor.classes = null key.serializer = class org.apache.kafka.common.serialization.StringSerializer linger.ms = 0 max.block.ms = 60000 max.in.flight.requests.per.connection = 5 max.request.size = 1048576 metadata.max.age.ms = 300000 metric.reporters = metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner receive.buffer.bytes = 32768 reconnect.backoff.max.ms = 10 reconnect.backoff.ms = 10 request.timeout.ms = 30000 retries = 0 retry.backoff.ms = 100 sasl.jaas.config = null sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = null sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.mechanism = GSSAPI security.protocol = PLAINTEXT send.buffer.bytes = 131072 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = null ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS transaction.timeout.ms = 60000 transactional.id = null value.serializer = class org.apache.kafka.common.serialization.StringSerializer [2019-08-27T11:24:15,127][INFO][org.apache.kafka.common.utils.AppInfoParser] Kafka version : 1.0.0 [2019-08-27T11:24:15,127][INFO][org.apache.kafka.common.utils.AppInfoParser] Kafka commitId : aaa7af6d4a11b29d [2019-08-27T11:24:15,178][INFO][logstash.inputs.tcp] Starting tcp input listener {:address=\>"0.0.0.0:3514", :ssl\_enable=\>"false"} [2019-08-27T11:24:15,856][INFO][logstash.pipeline] Pipeline started successfully {:pipeline\_id=\>"syslog", :thread=\>"#\"} [2019-08-27T11:24:15,866][INFO][logstash.inputs.udp] Starting UDP listener {:address=\>"0.0.0.0:3514"} [2019-08-27T11:24:15,914][INFO][logstash.inputs.udp] UDP listener started {:address=\>"0.0.0.0:3514", :receive\_buffer\_bytes=\>"106496", :queue\_size=\>"2000"} [2019-08-27T11:24:16,663][INFO][logstash.pipeline] Starting pipeline {:pipeline\_id=\>"winlogbeats", "pipeline.workers"=\>1, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50} [2019-08-27T11:24:16,708][INFO][org.apache.kafka.clients.producer.ProducerConfig] ProducerConfig values: acks = 1 batch.size = 16384 bootstrap.servers = [[slp-3zp.mydomain.com:9092](http://slp-3zp.mydomain.com:9092), [slp-b3c.mydomain.com:9092](http://slp-b3c.mydomain.com:9092)] buffer.memory = 33554432 client.id = compression.type = none connections.max.idle.ms = 540000 enable.idempotence = false interceptor.classes = null key.serializer = class org.apache.kafka.common.serialization.StringSerializer linger.ms = 0 max.block.ms = 60000 max.in.flight.requests.per.connection = 5 max.request.size = 1048576 metadata.max.age.ms = 300000 metric.reporters = metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner receive.buffer.bytes = 32768 reconnect.backoff.max.ms = 10 reconnect.backoff.ms = 10 request.timeout.ms = 30000 retries = 0 retry.backoff.ms = 100 sasl.jaas.config = null sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = null sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.mechanism = GSSAPI security.protocol = PLAINTEXT send.buffer.bytes = 131072 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = null ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS transaction.timeout.ms = 60000 transactional.id = null value.serializer = class org.apache.kafka.common.serialization.StringSerializer [2019-08-27T11:24:16,726][INFO][org.apache.kafka.common.utils.AppInfoParser] Kafka version : 1.0.0 [2019-08-27T11:24:16,727][INFO][org.apache.kafka.common.utils.AppInfoParser] Kafka commitId : aaa7af6d4a11b29d [2019-08-27T11:24:16,729][INFO][org.apache.kafka.clients.producer.ProducerConfig] ProducerConfig values: acks = 1 batch.size = 16384 bootstrap.servers = [[slp-3zp.mydomain.com:9092](http://slp-3zp.mydomain.com:9092), [slp-b3c.mydomain.com:9092](http://slp-b3c.mydomain.com:9092)] buffer.memory = 33554432 client.id = compression.type = none connections.max.idle.ms = 540000 enable.idempotence = false interceptor.classes = null key.serializer = class org.apache.kafka.common.serialization.StringSerializer linger.ms = 0 max.block.ms = 60000 max.in.flight.requests.per.connection = 5 max.request.size = 1048576 metadata.max.age.ms = 300000 metric.reporters = metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner receive.buffer.bytes = 32768 reconnect.backoff.max.ms = 10 reconnect.backoff.ms = 10 request.timeout.ms = 30000 retries = 0 retry.backoff.ms = 100 sasl.jaas.config = null sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = null sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.mechanism = GSSAPI security.protocol = PLAINTEXT send.buffer.bytes = 131072 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = null ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS transaction.timeout.ms = 60000 transactional.id = null value.serializer = class org.apache.kafka.common.serialization.StringSerializer ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 27, 2019, 4:32pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/14 "2019-08-27T16:32:44Z")
[2019-08-27T11:24:16,741][INFO][org.apache.kafka.common.utils.AppInfoParser] Kafka version : 1.0.0 [2019-08-27T11:24:16,741][INFO][org.apache.kafka.common.utils.AppInfoParser] Kafka commitId : aaa7af6d4a11b29d [2019-08-27T11:24:16,743][INFO][org.apache.kafka.clients.producer.ProducerConfig] ProducerConfig values: acks = 1 batch.size = 16384 bootstrap.servers = [[slp-3zp.mydomain.com:9092](http://slp-3zp.mydomain.com:9092), [slp-b3c.mydomain.com:9092](http://slp-b3c.mydomain.com:9092)] buffer.memory = 33554432 client.id = compression.type = none connections.max.idle.ms = 540000 enable.idempotence = false interceptor.classes = null key.serializer = class org.apache.kafka.common.serialization.StringSerializer linger.ms = 0 max.block.ms = 60000 max.in.flight.requests.per.connection = 5 max.request.size = 1048576 metadata.max.age.ms = 300000 metric.reporters = metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner receive.buffer.bytes = 32768 reconnect.backoff.max.ms = 10 reconnect.backoff.ms = 10 request.timeout.ms = 30000 retries = 0 retry.backoff.ms = 100 sasl.jaas.config = null sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = null sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.mechanism = GSSAPI security.protocol = PLAINTEXT send.buffer.bytes = 131072 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = null ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS transaction.timeout.ms = 60000 transactional.id = null value.serializer = class org.apache.kafka.common.serialization.StringSerializer [2019-08-27T11:24:16,762][INFO][org.apache.kafka.common.utils.AppInfoParser] Kafka version : 1.0.0 [2019-08-27T11:24:16,762][INFO][org.apache.kafka.common.utils.AppInfoParser] Kafka commitId : aaa7af6d4a11b29d [2019-08-27T11:24:17,152][INFO][logstash.inputs.beats] Beats inputs: Starting input listener {:address=\>"0.0.0.0:5044"} [2019-08-27T11:24:17,163][INFO][logstash.pipeline] Pipeline started successfully {:pipeline\_id=\>"winlogbeats", :thread=\>"#\"} [2019-08-27T11:24:17,169][INFO][logstash.agent] Pipelines running {:count=\>2, :pipelines=\>["syslog", "winlogbeats"]} [2019-08-27T11:24:17,170][INFO][org.logstash.beats.Server] Starting server on port: 5044 ---
**Author:** ![justkind](https://avatars.discourse-cdn.com/v4/letter/j/919ad9/32.png) [@justkind](https://discuss.elastic.co/u/justkind)\ **Post date:** [August 27, 2019, 5:02pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/15 "2019-08-27T17:02:07Z")
Happy to help Brian, hopefully we can figure out what's going on. We just want to line up logstash's input with whatever is being sent out from your log servers, whether it is filebeat log events (beats input), or generic events (your TCP and UDP inputs). If you're interested, you can check out the logstash syslog input plugin which might be useful as these plugins are made to specifically handle syslog events: [https://www.elastic.co/guide/en/logstash/6.2/plugins-inputs-syslog.html](https://www.elastic.co/guide/en/logstash/6.2/plugins-inputs-syslog.html) The logs for logstash you posted seem to show that it is properly setting up your TCP and UDP listeners on port 3514 and also has a beats input configured on port 5044. If you can, double check your syslogs is sending to the correct logstash on the correct port. For the filebeat log events, it seems there is an existing beats input configured somewhere so maybe try to look for that and see if there are any errors in the configuration. ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 27, 2019, 9:08pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/16 "2019-08-27T21:08:54Z")
my beats version changed from 6.2.2 to 6.8.2 after putting the version back, I am no longer seeing any ERROR in the log for filebeat. Still not seeing anything show in Kibana.... I've restarted logstash I've refreshed the index on Elastic heads with curl -XPOST '[http://localhost:9200/syslog-\*/\_refresh](http://localhost:9200/syslog-*/_refresh)' On my old syslog server I am using the same filebeat.yml contents for the new syslog server. Syslog data is its own drive mounted under /var/logs I use rsync on that partition to send data to the new syslog server. Syslog is a different version... maybe that is my problem? ---
**Author:** ![justkind](https://avatars.discourse-cdn.com/v4/letter/j/919ad9/32.png) [@justkind](https://discuss.elastic.co/u/justkind)\ **Post date:** [August 27, 2019, 9:39pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/17 "2019-08-27T21:39:07Z")
Hmm interesting that there were errors for the newer filebeats version, according to the [compatibility matrix](https://www.elastic.co/support/matrix#matrix_compatibility), all filebeat and logstash versions between 5.6.x-6.8.x should be fine. With no errors it's going to be hard to find the issue but we need to trace the flow of traffic. Could you clarify on the full flow? From the configuration you've shared, it appears to be: Filebeat -\> Logstash -\> Kafka -\> ? -\> Elasticsearch -\> Kibana Do you happen to have another Logstash between Kafka and Elasticsearch? Can you check anything on Kafka? Are indices populating in Elasticsearch? You can check on this by curl'ing your Elasticsearch with /\_cat/indices?v? We're just trying to figure out where traffic is stopping. If you have monitoring enabled, you should be able to see if logstash is processing events. ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 28, 2019, 6:15pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/18 "2019-08-28T18:15:11Z")
I might switch back to the newer 6.8.2 because it did provide more detail with error's... [(1)Syslog -\> Filebeat] -\> [(2)Logstash -\> Kafka -\> Zookeeper] -\> [(2)Elasticsearch] -\> [(1)Kibana] (According to our documentation) 1 server for syslog & filebeat 2 servers for logstash, kafka and zookeeper - HA PROXY 2 servers for Elasticsearch - HA PROXY 1 server for Kibana > [@justkind](#): > > Can you check anything on Kafka? Are indices populating in Elasticsearch? You can check on this by curl'ing your Elasticsearch with /\_cat/indices?v? Please be patient with me as I am still a noob when it comes to Elastic and friends. I can check logs in kafka, never looked at kafka before 😑 when I curl Elasticsearch I do get results - after removing the v? `curl -XGET 'http://slp-pkf.my.com:9200/_cat/indices?' | grep syslog-2019.08.28` ---
**Author:** ![justkind](https://avatars.discourse-cdn.com/v4/letter/j/919ad9/32.png) [@justkind](https://discuss.elastic.co/u/justkind)\ **Post date:** [August 29, 2019, 4:15pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/19 "2019-08-29T16:15:36Z")
Thanks for the information Brian, and no worries at all about experience level with tools. There's always someone more experienced/knowledgeable and also always someone with less knowledge/experience than you on any topic 🙂, I wouldn't call myself an expert either so would be great if someone on the @elastic team could chime in if they are reading this. The ?v URL parameter just adds column names so it's not important, but you stated that you are getting results if you hit the indices endpoint and grep for the syslog index with today's date. That should mean that you are indeed getting syslog data throughout your whole pipeline, if it's making its way throughout your entire flow to Elasticsearch. If you are not seeing it in Kibana, can you ensure that you have the correct index pattern? It has to match the index names you are seeing from that /\_cat/indices endpoint. If you already have an existing index pattern and it is not showing up any data, maybe you need to refresh the field list. You can do this by going to Management -\> Index Patterns -\> Click on your syslog index pattern -\> Click on the refresh button (looks like 2 circling arrows) on the top right and check again. Maybe delete the index pattern and create it again as a last resort. You can also confirm your indices have data in it by curl'ing Elasticsearch in a similar way to getting the indices. Assuming your syslog index is named exactly what you put in your grep statement: ``` curl -XGET 'http://slp-pkf.my.com:9200/syslog-2019.08.28/_search ``` And you should get results back, maybe change the date to today's date 2019.08.29 to ensure it is being updated constantly. ---
**Author:** ![Brian\_Tima](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_tima/32/50927_2.png) [@Brian\_Tima](https://discuss.elastic.co/u/Brian_Tima)\ **Post date:** [August 29, 2019, 7:23pm UTC](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993/20 "2019-08-29T19:23:28Z")
Thanks again for your help @justkind Yes I am getting data from the expiring central syslog server... [Next page](https://discuss.elastic.co/t/syslog-and-filebeat-assistance-request/192993.md?page=2)