Hi, We are getting repeated deserialization errors with Persistent Queues.
- Logstash 5.4.1 (unmodified CentOS image from elastic repos)
- 200GB SSD mounted disk for the queue
- Persistent queues enabled
pipeline.workers: 2 queue.type: persisted queue.max_bytes: 150gb
We set no other queue-related config (except stuff like http bind address), and we leave everything else to logstash defaults. The error starts to happen when Logstash is running. Please note, we run it in docker containers, so no runtime config reloads.
And once the error starts, well, that's it. It persists even after restarting Logstash. Pipeline is blocked, all subsequent events start buffering, but removing any single checkpoint/page file will also not work (will lead to the same seqnum mismatch error as in Must delete files in message queue to start logstash). We have to drop all the buffered logs.
Its exactly the same root error cause all the time:
Caused by: java.lang.IllegalArgumentException: No enum constant org.logstash.bivalues.BiValues.[B
We're now restarting Logstash on a daily basis, and we're losing logs with the PQ. We also have some backups of the page/checkpoint files, we can provide it if required. Can you please help?