I have configured a folder containing files which will be dynamically updated with some data which needs to be parsed by logstash and pushed into ES.
Here, whenever new data is written into any one of the input file, logstash is failing to detect and parse them(since db is also not updating). But if I do a restart of logstash it is able to detect the change and push the data into ES(since db is also updating).
It will be very helpful if someone can tell me how to make my logstash process the dynamic change happening in my configured file without requiring a restart of logstash.
There's a live preview panel for exactly this reasons.
Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.
That sounds like a bug to me, but not the bug you think it is. In read mode it is assumed that the files do not change. If the files grow you should be using tail mode.
The file input reads the file and does not expect it to grow, so it does not go back to it. However, when it restarts, it loads the sincedb, checks the file, sees the file is longer than the length in the sincedb and processes the delta. I would argue that it should not do that.