# Data missed in Logstash!

**URL:** https://discuss.elastic.co/t/data-missed-in-logstash/81904
**Category:** Logstash
**Created:** [April 11, 2017, 3:16am UTC](https://discuss.elastic.co/t/data-missed-in-logstash/81904 "2017-04-11T03:16:28Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![zhaochl](https://avatars.discourse-cdn.com/v4/letter/z/7ba0ec/32.png) [@zhaochl](https://discuss.elastic.co/u/zhaochl)
#### Post date: [April 11, 2017, 3:16am UTC](https://discuss.elastic.co/t/data-missed-in-logstash/81904/1 "2017-04-11T03:16:28Z")

</div>

Data missed a lot in logstash ,version 5.0, is it a serous bug ?

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 11, 2017, 4:59am UTC](https://discuss.elastic.co/t/data-missed-in-logstash/81904/2 "2017-04-11T04:59:40Z")

</div>

You will need to give more information about your configuration in order for anyone to be able to help you. How have you identified that there is data missing? What type of inputs are you using? Which version of Logstash are you using?

---

<div class="post-metadata">

### Author: ![zhaochl](https://avatars.discourse-cdn.com/v4/letter/z/7ba0ec/32.png) [@zhaochl](https://discuss.elastic.co/u/zhaochl)
#### Post date: [April 11, 2017, 6:21am UTC](https://discuss.elastic.co/t/data-missed-in-logstash/81904/3 "2017-04-11T06:21:08Z")

</div>

loogstash version 5.0  
I use kibana query log and view by hour, because of my log is rotate by hour, I compare the log line number by linux wc -l command to the kibana count , find that logs in kibana are missed, in all hour log files the last line of the file never read, and in some our files , kibana is mush less than the wc number of the orgginal logs.  
my logstash config is as follows:

input {  
file {  
path =\> ["/home/work/logs/stat.log\*"]  
sincedb\_path =\> "/home/work/logstash/bin/db/sincedb-access"  
codec=\> multiline {  
pattern =\> "^["  
negate =\> true  
what =\> "previous"  
}

}  
#stdin { }  
}

output:  
output {  
elasticsearch {  
hosts =\> ["172.17.0.33:8081","172.17.0.32:8081"]  
index =\> "stat-%{+YYYY.MM.dd}"  
}  
#stdout { codec =\> rubydebug }

}

thanks for any suggestion.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 11, 2017, 7:07am UTC](https://discuss.elastic.co/t/data-missed-in-logstash/81904/4 "2017-04-11T07:07:52Z")

</div>

If the [auto\_flush\_interval](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html#plugins-codecs-multiline-auto_flush_interval) parameter is not set for the multiline codec, the last line will never be released as there is no next line to trigger release on.

---

<div class="post-metadata">

### Author: ![zhaochl](https://avatars.discourse-cdn.com/v4/letter/z/7ba0ec/32.png) [@zhaochl](https://discuss.elastic.co/u/zhaochl)
#### Post date: [April 11, 2017, 10:03am UTC](https://discuss.elastic.co/t/data-missed-in-logstash/81904/5 "2017-04-11T10:03:22Z")

</div>

Yes It woks, thanks a lot  
but the log still lost a few ,is it normal for logstash lost several log when transporting the data

thanks a lot

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 9, 2017, 10:05am UTC](https://discuss.elastic.co/t/data-missed-in-logstash/81904/6 "2017-05-09T10:05:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
