If I want to scale my logstash (with PQ) horizontally in k8s, I would just increase the replica amount. Now that I have multiple nodes, I would like to understand how the PQ manages race conditions. I saw a .lock file but wasnt to sure if it answered all the use cases and additionally I was confused about why not use a volumeClaimTemplate (StatefulSet) for each replica/node therefore having no race condition.
I have one pipeline, but if I were to have more, there would be page.x and checkpoint files per pipeline or not, and if not so how do they manage race conditions?
I know this might be a nooby question, but I thank you regardless.
Probably not, these helm charts are created and maintained by bitnami, so you need to ask on a bitnami forum or something like that.
I have no experience with k8s, but if by replica you mean another pod running logstash, then yes, it is a different logstash node/instance and the instances are independent from each other and need different persistent queues.