Big gaps in recorded logs

I wonder if you are getting bit by nginx not really supporting logrotation. Most articles suggest a kill -USR1 in the script: https://www.loggly.com/docs/log-rotate/ and https://www.digitalocean.com/community/tutorials/how-to-configure-logging-and-log-rotation-in-nginx-on-an-ubuntu-vps

Side comments ...

Looking at your offset, the lag is what you are interested in. IMO 1000's are a little high to be in but if it kinda hangs around that then you are okay. Guessing from your Owner row it appears that you have two or three logstash instances that supply read from Kafka and submit the logs to Elasticsearch all with three threads. What type of boxes are those, specifically how many cores? You should try to make sure your threads match the number of cores for best performance, otherwise the threads will just fight each other to process when a single thread can still handle all the partitions.

Other thought, get those nginx logs into a json format so there is almost zero processing that has to happen for logstash and you get those timestamps synced up. Check out this post for some guidance: https://blog.pkhamre.com/logging-to-logstash-json-format-in-nginx/