# Not sending logs to logstash

**URL:** https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278
**Category:** Beats
**Tags:** filebeat
**Created:** [December 18, 2017, 4:42pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278 "2017-12-18T16:42:41Z")
**Posts on this page:** 19
**Page:** 1

<div class="post-metadata">

### Author: ![Nich240](https://avatars.discourse-cdn.com/v4/letter/n/df705f/32.png) [@Nich240](https://discuss.elastic.co/u/Nich240)
#### Post date: [December 18, 2017, 4:42pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/1 "2017-12-18T16:42:41Z")

</div>

We are having issue sending logs from filebeat to logstash. We do not see the network connections being made, nor errors. There is no firewall between the two components. Filebeat registry is showing files being ingested, but not sent. The logs don't show any errors. Logstash is set to receive a beats input at 5044. Filebeat is configured to connect at 5044. Any suggestions?

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [December 19, 2017, 4:17am UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/2 "2017-12-19T04:17:43Z")

</div>

Could you share logs, configs and versions of Logstash and Filebeat?

---

<div class="post-metadata">

### Author: ![Nich240](https://avatars.discourse-cdn.com/v4/letter/n/df705f/32.png) [@Nich240](https://discuss.elastic.co/u/Nich240)
#### Post date: [December 19, 2017, 3:23pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/3 "2017-12-19T15:23:05Z")

</div>

I updated to 6.1 yesterday. I had the same issue with 6.0

The filename that should be used is DataDomainFilesysStatus\_JSON\_201712191011

The date is in the filename (2017\_12\_19, 10:11am)

Filebeat: [https://pastebin.com/4PSzfeRg](https://pastebin.com/4PSzfeRg)  
Logstash: [https://pastebin.com/VqXndgcx](https://pastebin.com/VqXndgcx)  
filebeat.yml: [https://pastebin.com/Me1ysed1](https://pastebin.com/Me1ysed1)  
filter.conf: [https://pastebin.com/L8QzTHQX](https://pastebin.com/L8QzTHQX)

Thank You

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [December 22, 2017, 4:52am UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/4 "2017-12-22T04:52:29Z")

</div>

In the filebeat logs are the following errors: `ERR Failed to connect: dial tcp 127.0.0.1:5044: getsockopt: connection refused`. It seems Logstash is not reachable. Is it on localhost?

---

<div class="post-metadata">

### Author: ![Nich240](https://avatars.discourse-cdn.com/v4/letter/n/df705f/32.png) [@Nich240](https://discuss.elastic.co/u/Nich240)
#### Post date: [December 22, 2017, 11:53am UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/5 "2017-12-22T11:53:34Z")

</div>

Yes. All components are on the same host, at this time.

---

<div class="post-metadata">

### Author: ![Nich240](https://avatars.discourse-cdn.com/v4/letter/n/df705f/32.png) [@Nich240](https://discuss.elastic.co/u/Nich240)
#### Post date: [December 22, 2017, 4:53pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/6 "2017-12-22T16:53:34Z")

</div>

I see the connection problem that you mentioned in filebeat. I believe all services weren't running at that time. I restarted filebeat, and have uploaded the lastest to:  
[https://pastebin.com/XdzDx4iM](https://pastebin.com/XdzDx4iM)

---

<div class="post-metadata">

### Author: ![Nich240](https://avatars.discourse-cdn.com/v4/letter/n/df705f/32.png) [@Nich240](https://discuss.elastic.co/u/Nich240)
#### Post date: [December 26, 2017, 9:04pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/7 "2017-12-26T21:04:50Z")

</div>

Any ideas? I've restarted services, and rebooted. We don't see anything in the log. Has anyone else seen this issue?

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [December 28, 2017, 2:07am UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/8 "2017-12-28T02:07:04Z")

</div>

One note for the logstash log: Quite a few errors / deprecated notice show up because of the type usage. I wonder if you have some issue here.

From filebeat perspective the events are definitively confirmed to have been sent. On the LS side log unfortunately stops a soon as LS is ready to receive events. Can you remove the ES output and just us the stdout output to see if any events are flowing in. If you start LS in debug mode, you should also see more info.

---

<div class="post-metadata">

### Author: ![Nich240](https://avatars.discourse-cdn.com/v4/letter/n/df705f/32.png) [@Nich240](https://discuss.elastic.co/u/Nich240)
#### Post date: [December 28, 2017, 4:43pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/9 "2017-12-28T16:43:56Z")

</div>

Ruflin,

Where do we remove the type usage?  
I have removed the ES part of the configuration file, and still see the error.  
I am running logstash in Debug mode:  
/usr/share/logstash/bin/logstash --path.settings /etc/logstash --path.config /etc/logstash/conf.d/

At 11:30am, I restarted the services:  
filebeat: [https://pastebin.com/R23BbBVb](https://pastebin.com/R23BbBVb)  
logstash: [https://pastebin.com/R74C0VRT](https://pastebin.com/R74C0VRT)  
logstash/conf.d/filter.conf: [https://pastebin.com/En4Vnrmh](https://pastebin.com/En4Vnrmh)

Any other ideas? Thank you.

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [January 3, 2018, 11:05pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/10 "2018-01-03T23:05:33Z")

</div>

That is pretty strange as I would not expect this message when you remove the elasticsearch output.

Ignoring the error for now, do you see the events coming into LS?

---

<div class="post-metadata">

### Author: ![Nich240](https://avatars.discourse-cdn.com/v4/letter/n/df705f/32.png) [@Nich240](https://discuss.elastic.co/u/Nich240)
#### Post date: [January 5, 2018, 9:29pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/11 "2018-01-05T21:29:24Z")

</div>

I still don't see events. Filebeat sees the file change, but logstash doesn't see it. Any ideas?

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [January 9, 2018, 4:52am UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/12 "2018-01-09T04:52:10Z")

</div>

Could you run filebeat with the debug log enabled and share it?

---

<div class="post-metadata">

### Author: ![Nich240](https://avatars.discourse-cdn.com/v4/letter/n/df705f/32.png) [@Nich240](https://discuss.elastic.co/u/Nich240)
#### Post date: [January 9, 2018, 8:26pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/13 "2018-01-09T20:26:18Z")

</div>

It looks like we got a lot more information from the debug.  
Logstash - [https://pastebin.com/NVuuUy7b](https://pastebin.com/NVuuUy7b)  
filebeat.degub - [https://pastebin.com/gfsy1Zyb](https://pastebin.com/gfsy1Zyb)  
filebeat - [https://zerobin.net/?bdb30e7c26eecd50#rJqNQDDlwsgRrB8oPyY1zSSblh/f2p26m3vzRxwZ12g=](https://zerobin.net/?bdb30e7c26eecd50#rJqNQDDlwsgRrB8oPyY1zSSblh/f2p26m3vzRxwZ12g=)

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [January 12, 2018, 2:26am UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/14 "2018-01-12T02:26:41Z")

</div>

Can you share the content of for example `/mnt/syslogs/Storage/Reports/DataDomain/FilesysStatus/DataDomainFilesysStatus_JSON_201712270530.txt` Do your files have a new line at the end?

---

<div class="post-metadata">

### Author: ![Nich240](https://avatars.discourse-cdn.com/v4/letter/n/df705f/32.png) [@Nich240](https://discuss.elastic.co/u/Nich240)
#### Post date: [January 12, 2018, 3:08pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/15 "2018-01-12T15:08:46Z")

</div>

I have attached my JSON files. One file is from 12/27. The other is more recent 1/12.

DataDomainFilesysStatus\_JSON\_201801120530.txt  
[https://pastebin.com/ktkqDZtu](https://pastebin.com/ktkqDZtu)

DataDomainFilesysStatus\_JSON\_201712270530.txt  
[https://pastebin.com/97ZKnNgb](https://pastebin.com/97ZKnNgb)

---

<div class="post-metadata">

### Author: ![ReddyRajanala](https://avatars.discourse-cdn.com/v4/letter/r/3be4f8/32.png) [@ReddyRajanala](https://discuss.elastic.co/u/ReddyRajanala)
#### Post date: [January 15, 2018, 4:19pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/16 "2018-01-15T16:19:30Z")

</div>

Can you verify logstash logs to see if there are errors reaching out to elasticsearch? You may have to specify complete host url to elasticsearch instead of localhost:9200 in your logstash pipeline configuration

---

<div class="post-metadata">

### Author: ![Nich240](https://avatars.discourse-cdn.com/v4/letter/n/df705f/32.png) [@Nich240](https://discuss.elastic.co/u/Nich240)
#### Post date: [January 15, 2018, 7:25pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/17 "2018-01-15T19:25:57Z")

</div>

I have noticed that my regular file seems to work, but the JSON didn't. I have added a newline character to the JSON file, and it is being passed to logstash.

---

<div class="post-metadata">

### Author: ![Nich240](https://avatars.discourse-cdn.com/v4/letter/n/df705f/32.png) [@Nich240](https://discuss.elastic.co/u/Nich240)
#### Post date: [January 18, 2018, 9:14pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/18 "2018-01-18T21:14:49Z")

</div>

With the new line character, this issue is resolved. WE can close this.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 15, 2018, 9:15pm UTC](https://discuss.elastic.co/t/not-sending-logs-to-logstash/112278/19 "2018-02-15T21:15:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
