File not loaded entirely using stdin plugin

[_kml]

I am using stdin plugin to load a file of about 2500 events, parse it and then write it to elasticsearch, so what I have is:
$ cat file | ./bin/logstash -f config/myconfig.conf

but what I have is that only about 600 events only are written to elasticsearch.

Afetr that, when I run the command:

$ ./bin/logstash -f config/myconfig.conf

the remaining part of the file is then handled.

do I have a problem with buffering? Can anyone help or expalin what is the problem?

[_kml]

The problem is: I had in my logstash.yml:
queue.type: persisted
the default value is memory, when I commented this, it worked.

[yaauie (Ry Biesemeyer)]

As a folow-up, enabling queue.drain is a way to resolve this with the persistent queue enabled.

Here's why:

• when the stdin input reaches EOF, it shuts itself down
• when a pipeline has no active inputs, it initiates shutdown
• the memory queue, and the persistent queue in drain-mode, hold the pipeline open until all wueued events are processed
• the persistent queue without drain-mode allows the pipeline to shut down immediately, since the event's have safely been spooled to disk.