# Logstash does not send few log events generally the last ones
**URL:** https://discuss.elastic.co/t/logstash-does-not-send-few-log-events-generally-the-last-ones/285802
**Category:** Logstash
**Created:** [October 4, 2021, 10:16am UTC](https://discuss.elastic.co/t/logstash-does-not-send-few-log-events-generally-the-last-ones/285802 "2021-10-04T10:16:57Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Pratyush\_Rath](https://discuss.elastic.co/u/Pratyush_Rath)
#### Post date: [October 4, 2021, 10:16am UTC](https://discuss.elastic.co/t/logstash-does-not-send-few-log-events-generally-the-last-ones/285802/1 "2021-10-04T10:16:57Z")
I am using an ELK stack in my local machine . I have tested sending log events from variety of inputs like file, stdin, filebeat and outputs like Elasticsearch, stdout and tried variety of combinations but the Outputs always has logs missing mostly the last event of the file.
I have also tested without using any filter and the problem still persists.
Sometimes when I close the logstash using Ctrl + C only then it sends only one of the log not previously sent.
I am on Windows and all ELK versions are 7.15.0
How to ensure all the logs from all the input sources are sent to Elasticsearch ?
---
### Author:  [@FALEN](https://discuss.elastic.co/u/FALEN)
#### Post date: [October 4, 2021, 11:29am UTC](https://discuss.elastic.co/t/logstash-does-not-send-few-log-events-generally-the-last-ones/285802/2 "2021-10-04T11:29:32Z")
Similar issue was occurring because multiline codec was used for input. Could you paste the config here please?
The multiline codec will collapse multiline messages and merge them into a single event.
> If you are using a Logstash input plugin that supports multiple hosts, such as the [beats input plugin](https://www.elastic.co/guide/en/logstash/7.15/plugins-inputs-beats.html), you should not use the multiline codec to handle multiline events. Doing so may result in the mixing of streams and corrupted event data. In this situation, you need to handle multiline events before sending the event data to Logstash.
> [Multiline codec plugin | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html)
---
### Author:  [@Pratyush\_Rath](https://discuss.elastic.co/u/Pratyush_Rath)
#### Post date: [October 4, 2021, 11:38am UTC](https://discuss.elastic.co/t/logstash-does-not-send-few-log-events-generally-the-last-ones/285802/3 "2021-10-04T11:38:53Z")
> [@FALEN](#):
>
> ugin that
```auto
input {
file{
path => "C:/Users/pratyush.rath/Desktop/Elasticsearch3/Testing/logs/*"
start_position => "beginning"
sincedb_path => "NUL"
}
}
filter {
}
output {
stdout{}
}
```
Earlier I was using multiline. But when I found the problem , I tried for normal events but still no progress.
---
### Author:  [@FALEN](https://discuss.elastic.co/u/FALEN)
#### Post date: [October 4, 2021, 11:56am UTC](https://discuss.elastic.co/t/logstash-does-not-send-few-log-events-generally-the-last-ones/285802/4 "2021-10-04T11:56:13Z")
> [@Pratyush\_Rath](#):
>
> `sincedb_path => "NUL"`
I'm not sure if its related but sincedb\_path should be a filepath, maybe you can try without this setting
#### `sincedb_path` [edit](https://github.com/logstash-plugins/logstash-input-file/edit/master/docs/index.asciidoc)
- Value type is [string](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#string)
- There is no default value for this setting.
Path of the sincedb database file (keeps track of the current position of monitored log files) that will be written to disk. The default will write sincedb files to `/plugins/inputs/file` NOTE: it must be a file path and not a directory path
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 4, 2021, 12:29pm UTC](https://discuss.elastic.co/t/logstash-does-not-send-few-log-events-generally-the-last-ones/285802/5 "2021-10-04T12:29:14Z")
@Pratyush_Rath, If it is the last event of the file that it is not being sent, then the reason is probably caused by the fact that this event doesn't end in a line break.
Losgtash (and filebeat) uses line break characters to know when an event endend, if the last line does not have this character, it will not be seen as an event and will not be sent.
How are the files created? Depending on how the files are created you can change the default reading mode from `tail` to `read`. The `tail` mode tracks the file for new changes, and the last line will only be sent if it also has a line break character in the end, the `read` mode will read the entire file until EOF. The [documentation](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#_description_13) describes how this work.
@FALEN, in this case the `NUL` is the windows equivalent of `/dev/null`, so `sincedb_path => "NUL"` means that the input won't use sincedb and the file will be reread every time logstash is restarted.
---
### Author:  [@Pratyush\_Rath](https://discuss.elastic.co/u/Pratyush_Rath)
#### Post date: [October 4, 2021, 1:40pm UTC](https://discuss.elastic.co/t/logstash-does-not-send-few-log-events-generally-the-last-ones/285802/6 "2021-10-04T13:40:14Z")
Thankyou @leandrojmp.
When there are lot of files to tail at the same time (e.g. I have lots of files in a folder which is input for logstash) it missed some of the small files.
Can this be solved in logstash itself or should I use filebeat? Because I am using multiline codec and it is recommended in documentation.
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 4, 2021, 1:44pm UTC](https://discuss.elastic.co/t/logstash-does-not-send-few-log-events-generally-the-last-ones/285802/7 "2021-10-04T13:44:28Z")
It depends on how the files are being written.
Are they being constantly written by another application or when they are written just once? If they are written just once you can change the reading mode of the logstash file input, if they are constantly being written you will need to use the default mode, which is tail.
As I said, both logstash an filebeat needs an line break character in the end of each line to know that the event has ended.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 1, 2021, 1:45pm UTC](https://discuss.elastic.co/t/logstash-does-not-send-few-log-events-generally-the-last-ones/285802/8 "2021-11-01T13:45:26Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.