Logstash to parse only part of apache log file

Elastic Stack Logstash
1

Hi Guys,

I am re-indexing few ones. They are monthly.

I deleted the a month index say Oct 2018 one.

Oct Index contains data :
Oct 1st UTC - Oct 31 UTC

But my apache logs are in PST and are rotated daily. So I need to put log files from "Sep 30 5 PM PST - Oct 30 4:59 PM PST" for Oct ELK Index (UTC).

Is there a way to make logstash to parse only portion of log file instead of complete file, so that I can pass Sep 30 PST and Oct 30 PST log files and can ask logstatsh to pick from/till time specified.

Thanks

2

No, I do not think that is possible, but you might be able to use a drop filter to avoid processing data outside a specific interval.

3

So what is the efficient way for my issue and also to regenerate indexes ...

4

Hi Christian

Is there a way to monitor how much of a log file is parsed by logstash. As during downtimes of systems, where we have to stop logstash and es running processes, are they re-runnable. I mean they will start again where they left. Please suggest how to handle ELK processing during/post downtimes.

Thanks

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.