# Filebeat not sending logs to Elasticsearch

**URL:** <https://discuss.elastic.co/t/filebeat-not-sending-logs-to-elasticsearch/52925>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [June 15, 2016, 10:18pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-logs-to-elasticsearch/52925 "2016-06-15T22:18:19Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![keya](https://avatars.discourse-cdn.com/v4/letter/k/8baadc/32.png) [@keya](https://discuss.elastic.co/u/keya)\
**Post date:** [June 15, 2016, 10:18pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-logs-to-elasticsearch/52925/1 "2016-06-15T22:18:19Z")

</div>

Hi,  
I am new to Filebeat and cannot get the logs to reach Elasticsearch. I have Filebeat and Elasticsearch on the same Ubuntu instance.

Below is a snapshot from the filebeat logs:

2016-06-15T14:38:00-07:00 DBG Disable stderr logging  
2016-06-15T14:38:00-07:00 INFO Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]  
2016-06-15T14:38:00-07:00 DBG filters:  
2016-06-15T14:38:00-07:00 INFO Setup Beat: filebeat; Version: 5.0.0-alpha3  
2016-06-15T14:38:00-07:00 DBG Initializing output plugins  
2016-06-15T14:38:00-07:00 INFO Loading template enabled. Reading template file: /etc/filebeat/filebeat.template.json  
2016-06-15T14:38:00-07:00 INFO Elasticsearch url: [http://localhost:9200](http://localhost:9200)  
2016-06-15T14:38:00-07:00 INFO Activated elasticsearch as output plugin.  
2016-06-15T14:38:00-07:00 DBG Create output worker  
2016-06-15T14:38:00-07:00 DBG No output is defined to store the topology. The server fields might not be filled.  
2016-06-15T14:38:00-07:00 INFO Publisher name: ubuntu  
2016-06-15T14:38:00-07:00 INFO Flush Interval set to: 1s  
2016-06-15T14:38:00-07:00 INFO Max Bulk Size set to: 50  
2016-06-15T14:38:00-07:00 DBG create bulk processing worker (interval=1s, bulk size=50)  
2016-06-15T14:38:00-07:00 INFO filebeat start running.  
2016-06-15T14:38:00-07:00 INFO Registry file set to: /var/lib/filebeat/registry  
2016-06-15T14:38:00-07:00 INFO Loading registrar data from /var/lib/filebeat/registry  
2016-06-15T14:38:00-07:00 DBG Error decoding old state: json: cannot unmarshal array into Go value of type map[string]input.FileState  
2016-06-15T14:38:00-07:00 INFO States Loaded from registrar: 3  
2016-06-15T14:38:00-07:00 DBG Spooler will use the default spool\_size of 2048  
2016-06-15T14:38:00-07:00 DBG Spooler will use the default idle\_timeout of 5s  
2016-06-15T14:38:00-07:00 INFO Loading Prospectors: 1  
2016-06-15T14:38:00-07:00 INFO buffer\_size set to: 16384  
2016-06-15T14:38:00-07:00 INFO document\_type set to: log  
2016-06-15T14:38:00-07:00 INFO input\_type set to: log  
2016-06-15T14:38:00-07:00 INFO Set backoff duration to 1s  
2016-06-15T14:38:00-07:00 INFO backoff\_factor set to: 2  
2016-06-15T14:38:00-07:00 INFO Set max\_backoff duration to 10s  
2016-06-15T14:38:00-07:00 INFO force\_close\_file is disabled  
2016-06-15T14:38:01-07:00 INFO Set close\_older duration to 1h0m0s  
2016-06-15T14:38:01-07:00 INFO max\_bytes set to: 10485760  
2016-06-15T14:38:01-07:00 DBG exclude\_files: []  
2016-06-15T14:38:01-07:00 INFO Load previous states from registry into memory  
2016-06-15T14:38:01-07:00 INFO Previous states loaded: 3  
2016-06-15T14:38:01-07:00 DBG File Configs: [/home/kshah/Downloads/logs/log3.log]  
2016-06-15T14:38:01-07:00 INFO Loading Prospectors completed. Number of prospectors: 1  
2016-06-15T14:38:01-07:00 INFO All prospectors are initialised and running with 3 states to persist  
2016-06-15T14:38:01-07:00 INFO Start sending events to output  
2016-06-15T14:38:01-07:00 INFO Starting Registrar  
2016-06-15T14:38:01-07:00 INFO Starting spooler: spool\_size: 2048; idle\_timeout: 5s  
2016-06-15T14:38:01-07:00 DBG Starting prospector 0  
2016-06-15T14:38:01-07:00 INFO Starting prospector of type: log  
2016-06-15T14:38:01-07:00 DBG Start next scan  
2016-06-15T14:38:01-07:00 DBG Check file for harvesting: /home/kshah/Downloads/logs/log3.log  
2016-06-15T14:38:01-07:00 DBG Update existing file for harvesting: /home/kshah/Downloads/logs/log3.log, offset: 446  
2016-06-15T14:38:01-07:00 DBG Resuming harvesting of file: /home/kshah/Downloads/logs/log3.log, offset: 446  
2016-06-15T14:38:01-07:00 DBG harvest: "/home/kshah/Downloads/logs/log3.log" position:446 (offset snapshot:0)  
2016-06-15T14:38:01-07:00 INFO Harvester started for file: /home/kshah/Downloads/logs/log3.log  
2016-06-15T14:38:01-07:00 DBG Update state: /home/kshah/Downloads/logs/log3.log, offset: 446  
2016-06-15T14:38:01-07:00 DBG Old state overwritten for /home/kshah/Downloads/logs/log3.log  
2016-06-15T14:38:01-07:00 DBG End of file reached: /home/kshah/Downloads/logs/log3.log; Backoff now.  
2016-06-15T14:38:02-07:00 DBG End of file reached: /home/kshah/Downloads/logs/log3.log; Backoff now.  
2016-06-15T14:38:04-07:00 DBG End of file reached: /home/kshah/Downloads/logs/log3.log; Backoff now.  
2016-06-15T14:38:06-07:00 DBG Flushing spooler because of timeout. Events flushed: 1  
2016-06-15T14:38:06-07:00 DBG No events to publish  
2016-06-15T14:38:06-07:00 INFO Events sent: 1  
2016-06-15T14:38:06-07:00 DBG Processing 1 events  
2016-06-15T14:38:06-07:00 DBG Old state overwritten for /home/kshah/Downloads/logs/log3.log  
2016-06-15T14:38:06-07:00 DBG Write registry file: /var/lib/filebeat/registry  
2016-06-15T14:38:06-07:00 INFO Registry file updated. 3 states written.  
2016-06-15T14:38:08-07:00 DBG End of file reached: /home/kshah/Downloads/logs/log3.log; Backoff now.

Has anyone faced this issue? Any help is appreciated.

Thanks

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [June 16, 2016, 10:57am UTC](https://discuss.elastic.co/t/filebeat-not-sending-logs-to-elasticsearch/52925/2 "2016-06-16T10:57:51Z")

</div>

It seems like no events are sent, as filebeat continues at a previous position. I assume you started filebeat before. Make sure to stop filebeat, remove the registry file and start it again so it starts sending from scratch.

---

<div class="post-metadata">

**Author:** ![keya](https://avatars.discourse-cdn.com/v4/letter/k/8baadc/32.png) [@keya](https://discuss.elastic.co/u/keya)\
**Post date:** [June 16, 2016, 6:01pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-logs-to-elasticsearch/52925/3 "2016-06-16T18:01:10Z")

</div>

Thanks. That worked.  
Do I need to remove it everytime? what is the significance of registry file?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 17, 2016, 3:38am UTC](https://discuss.elastic.co/t/filebeat-not-sending-logs-to-elasticsearch/52925/4 "2016-06-17T03:38:23Z")

</div>

The registry file controls where the process reads from each time it opens the file.  
You will only need to do this if you want to reprocess the file.

If you are still testing, then try setting the file to `/dev/null`, then it'll never create one.

---

<div class="post-metadata">

**Author:** ![keya](https://avatars.discourse-cdn.com/v4/letter/k/8baadc/32.png) [@keya](https://discuss.elastic.co/u/keya)\
**Post date:** [June 17, 2016, 3:19pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-logs-to-elasticsearch/52925/5 "2016-06-17T15:19:17Z")

</div>

Ok. Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2016, 10:18pm UTC](https://discuss.elastic.co/t/filebeat-not-sending-logs-to-elasticsearch/52925/6 "2016-07-06T22:18:28Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.
