# Configure Filebeat for All Logs

**URL:** <https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 19, 2016, 8:53am UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514 "2016-01-19T08:53:16Z")\
**Posts on this page:** 16\
**Page:** 2

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 25, 2016, 11:50am UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/22 "2016-01-25T11:50:03Z")

</div>

Yes, that's the problem. Why isn't Filebeat able to connect to localhost:9200? Or should it be connecting to something else?

---

<div class="post-metadata">

**Author:** ![naseerudeenmeeajane](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@naseerudeenmeeajane](https://discuss.elastic.co/u/naseerudeenmeeajane)\
**Post date:** [January 25, 2016, 2:04pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/23 "2016-01-25T14:04:11Z")

</div>

I think I make some mistake.  
Filebeat is installed on a remote server, so it's normal that there is nothing on localhost.  
I need to configure filebeat to listen on my remote server logstash?

---

<div class="post-metadata">

**Author:** ![naseerudeenmeeajane](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@naseerudeenmeeajane](https://discuss.elastic.co/u/naseerudeenmeeajane)\
**Post date:** [January 25, 2016, 2:17pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/24 "2016-01-25T14:17:23Z")

</div>

Ok my mistake. When I reconfigure Filebeat I have deleted my old file  
Will make modification and will see if all is ok

---

<div class="post-metadata">

**Author:** ![naseerudeenmeeajane](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@naseerudeenmeeajane](https://discuss.elastic.co/u/naseerudeenmeeajane)\
**Post date:** [January 26, 2016, 9:58am UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/25 "2016-01-26T09:58:14Z")

</div>

Hello,

Modification has been made. Now my elasticsearch (on a remote server) is listening on the good interface.  
Then I configure kibana to fetch the good server for Elasticsearch.

Also, modification has been made on filebeat. Here is the a part of the log file :

> 2016-01-26T10:48:33+01:00 DBG %!s(int=0) events out of %!s(int=200) events sent to logstash. Continue sending ...  
> 2016-01-26T10:48:33+01:00 INFO Error publishing events (retrying): EOF  
> 2016-01-26T10:48:33+01:00 INFO send fail  
> 2016-01-26T10:48:33+01:00 INFO backoff retry: 1s  
> 2016-01-26T10:48:34+01:00 DBG Try to publish %!s(int=200) events to logstash with window size %!s(int=10)  
> 2016-01-26T10:48:34+01:00 DBG %!s(int=0) events out of %!s(int=200) events sent to logstash. Continue sending ...  
> 2016-01-26T10:48:34+01:00 INFO Error publishing events (retrying): EOF  
> 2016-01-26T10:48:34+01:00 INFO send fail  
> 2016-01-26T10:48:34+01:00 INFO backoff retry: 2s  
> 2016-01-26T10:48:38+01:00 DBG Start next scan  
> 2016-01-26T10:48:38+01:00 DBG scan path /var/log/syslog  
> 2016-01-26T10:48:38+01:00 DBG Check file for harvesting: /var/log/syslog  
> 2016-01-26T10:48:38+01:00 DBG Update existing file for harvesting: /var/log/syslog  
> 2016-01-26T10:48:38+01:00 DBG Not harvesting, file didn't change: /var/log/syslog

So I think for now Filebeat is Ok ?

Do you have info avout ELK ? Kibana is my web interface to see informations from logstash ? Logstash is the applicative that collect info from log ? And what about Elasticsearch ?

Thanks

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 26, 2016, 12:49pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/26 "2016-01-26T12:49:09Z")

</div>

> So I think for now Filebeat is Ok ?

The log indicates that there's an error, at least a temporary one. Does the data eventually reach Logstash and Elasticsearch?

> Do you have info avout ELK ? Kibana is my web interface to see informations from logstash ?

Yes.

> Logstash is the applicative that collect info from log ?

Yes.

> And what about Elasticsearch ?

ES stores the data and performs the queries.

---

<div class="post-metadata">

**Author:** ![naseerudeenmeeajane](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@naseerudeenmeeajane](https://discuss.elastic.co/u/naseerudeenmeeajane)\
**Post date:** [January 26, 2016, 1:26pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/27 "2016-01-26T13:26:10Z")

</div>

Do you know How can I see if datas have reached Logstash or ES ?  
From what I see in Kibana, some datas have been reached as I have some informations on Kibana but now no new data

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 26, 2016, 1:28pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/28 "2016-01-26T13:28:29Z")

</div>

Are there additional indications in the log that Filebeat succeeds or fails to send events to Logstash? As I said earlier, the log snippet you posted earlier that Filebeat had problems but that it would try again. Which version of Filebeat is this?

---

<div class="post-metadata">

**Author:** ![naseerudeenmeeajane](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@naseerudeenmeeajane](https://discuss.elastic.co/u/naseerudeenmeeajane)\
**Post date:** [January 26, 2016, 2:12pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/29 "2016-01-26T14:12:55Z")

</div>

HEre is the version of Filebeat :

> root@crinforecettevm01:/var/log/mybeat# filebeat --version  
> filebeat version 1.0.1 (amd64)  
> root@crinforecettevm01:/var/log/mybeat#

When looking into log of filebeat, here is an error :

> 2016-01-26T15:07:48+01:00 INFO Connecting error publishing events (retrying): read tcp 172.31.0.168:44959-\>37.59.235.27:5044: i/o timeout

May be I need to configure something on my remote server ( logstash ?)

---

<div class="post-metadata">

**Author:** ![naseerudeenmeeajane](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@naseerudeenmeeajane](https://discuss.elastic.co/u/naseerudeenmeeajane)\
**Post date:** [January 26, 2016, 2:18pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/30 "2016-01-26T14:18:14Z")

</div>

Just to be sure, here is my infra :

Server 1 : Elasticsearch + Logstash + Kibana  
Server 2 : Filebeat

Filebeat send info on my remote server.

May be I need filebeat on my remote server ? As in the error below, I see

> read tcp 172.31.0.168:44959-\>37.59.235.27:5044: i/o timeout

But nothing running on port 5044 !  
37.59.235.27 is server 1  
172.31.0.168 is server 2

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 26, 2016, 2:34pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/31 "2016-01-26T14:34:06Z")

</div>

> 2016-01-26T15:07:48+01:00 INFO Connecting error publishing events (retrying): read tcp 172.31.0.168:44959-\>37.59.235.27:5044: i/o timeout

This is a network connectivity issue. Is there e.g. a firewall blocking access from your Filebeat host (172.31.0.168) to your Logstash host (37.59.235.27)?

---

<div class="post-metadata">

**Author:** ![naseerudeenmeeajane](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@naseerudeenmeeajane](https://discuss.elastic.co/u/naseerudeenmeeajane)\
**Post date:** [January 26, 2016, 3:12pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/32 "2016-01-26T15:12:33Z")

</div>

Ok thanks. I will see this with the right service.  
Will let you know ASAP

Thank you so much for all your help

---

<div class="post-metadata">

**Author:** ![naseerudeenmeeajane](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@naseerudeenmeeajane](https://discuss.elastic.co/u/naseerudeenmeeajane)\
**Post date:** [January 26, 2016, 3:17pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/33 "2016-01-26T15:17:48Z")

</div>

Just a question, on the remote server nothing is running on port 5044. Is it normal?

---

<div class="post-metadata">

**Author:** ![naseerudeenmeeajane](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@naseerudeenmeeajane](https://discuss.elastic.co/u/naseerudeenmeeajane)\
**Post date:** [January 26, 2016, 3:24pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/34 "2016-01-26T15:24:19Z")

</div>

Juste check my conf and it seems it's ok on logstash side.  
And no issue from my firewall as when making "telnet 37.59.235.27 5044" all is ok

May be I need to configure my logstash a little bit better ?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [January 26, 2016, 4:51pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/35 "2016-01-26T16:51:01Z")

</div>

firewall/nat/whatever having timeout on connections? Sometimes firewalls silently close connections without sending FIN which kinda breaks communication.

---

<div class="post-metadata">

**Author:** ![naseerudeenmeeajane](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@naseerudeenmeeajane](https://discuss.elastic.co/u/naseerudeenmeeajane)\
**Post date:** [January 28, 2016, 12:22pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/36 "2016-01-28T12:22:12Z")

</div>

Hello

Here is the error on filebeat :

> 2016-01-28T13:19:03+01:00 INFO Error publishing events (retrying): EOF

I make the necessary on firewall side

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:56pm UTC](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514/37 "2017-07-05T21:56:12Z")

</div>

Closed on July 5, 2017, 9:56pm UTC



[Previous page](https://discuss.elastic.co/t/configure-filebeat-for-all-logs/39514.md?page=1)
