I've been doing parsing with Logstash on my new machine in the exact same way as I did in the past, but Logstash is constantly losing connection to Elastic this time ("Attempted to send bulk request but there are no living connections in the connection pool"). It fixes itself after waiting maybe 5-10 minutes, but this happens every now and then and is probably massively slowing down the parsing speed.
What has changed:
- Elastic Stack upgraded from 6.24 -> 6.30. I heard that one change is the timeout setting changed, but I am not sure why/how this will affect my configuration since I haven't changed much of anything else.
- Memory settings for Elastic, as I now give it 31GB ram (up from 8GB).