# Filebeat reads 1 event log multiple times

**URL:** <https://discuss.elastic.co/t/filebeat-reads-1-event-log-multiple-times/81822>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 10, 2017, 1:16pm UTC](https://discuss.elastic.co/t/filebeat-reads-1-event-log-multiple-times/81822 "2017-04-10T13:16:04Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![vishalkin](https://avatars.discourse-cdn.com/v4/letter/v/ccd318/32.png) [@vishalkin](https://discuss.elastic.co/u/vishalkin)\
**Post date:** [April 10, 2017, 1:16pm UTC](https://discuss.elastic.co/t/filebeat-reads-1-event-log-multiple-times/81822/1 "2017-04-10T13:16:04Z")

</div>

I have a Elasticsearch, Logstash, Kibana, Filebeat (5.1.1) running on same server.  
When I load a json file in the file beat and try and see the output in Kibana, I see same event\_log is added 4 times in Elastic index.

### Here is my filebeat.yml

> filebeat.prospectors:

> ```
> - input_type: log 
> 
> ```

> paths:  
> - /path\_to\_json\_log\_file/\*.json  
> exclude\_files: [".gz$"]  
> tag: ["JSON"]

> scan\_frequency: 1s  
> close\_inactive: 1m  
> clean\_removed: true

> multiline.pattern: ^{

> multiline.negate: true

> multiline.match: after

> document\_type: JSON

> output.logstash:
> 
> # The Logstash hosts
> 
> hosts: ["localhost:5045"]

> output.console:  
> pretty: true  
> logging.level: debug

### Logstash config file

> input {  
> beats{  
> port =\> 5045  
> }

> }

> filter {  
> json{  
> source =\> message  
> }  
> date{  
> match =\> ["timeMillis","UNIX\_MS"]  
> target =\> "@timestamp"  
> }  
> }   
> output  
> {  
> stdout { codec =\> json\_lines }  
> elasticsearch {   
> index =\> "ap-index"  
> hosts =\> ["localhost:9222"]  
> }  
> }

### JSON Format:

> {"timeMillis":1491826809179,"thread":"main","level":"INFO","loggerName":"some logger","message":"Data load Driver","endOfBatch":true,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","contextMap":{"hostName":"some hostn\_name","instanceName":"node3"},"threadId":1,"threadPriority":5}

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [April 13, 2017, 11:42am UTC](https://discuss.elastic.co/t/filebeat-reads-1-event-log-multiple-times/81822/2 "2017-04-13T11:42:00Z")

</div>

Which filebeat version are you using? Could you share your filebeat log output? Could you try to write to file and see if one event still ends up in multiple times in the output?

---

<div class="post-metadata">

**Author:** ![vishalkin](https://avatars.discourse-cdn.com/v4/letter/v/ccd318/32.png) [@vishalkin](https://discuss.elastic.co/u/vishalkin)\
**Post date:** [April 13, 2017, 12:31pm UTC](https://discuss.elastic.co/t/filebeat-reads-1-event-log-multiple-times/81822/3 "2017-04-13T12:31:22Z")

</div>

I am using Filebeat 5.1.1  
filebeat log output says it does not receive an acknowledgement.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [April 13, 2017, 12:46pm UTC](https://discuss.elastic.co/t/filebeat-reads-1-event-log-multiple-times/81822/4 "2017-04-13T12:46:54Z")

</div>

Then there is probably the problem. Can you share the log output?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 13, 2017, 2:37pm UTC](https://discuss.elastic.co/t/filebeat-reads-1-event-log-multiple-times/81822/5 "2017-04-13T14:37:56Z")

</div>

Have you tried to upgrade the logstash-input-beats plugin?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 11, 2017, 2:51pm UTC](https://discuss.elastic.co/t/filebeat-reads-1-event-log-multiple-times/81822/6 "2017-05-11T14:51:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.