Logstash-filter-zeromq is timing out with ZeroMQ

[anmoljagetia]

 zeromq {
        address => ["tcp://10.6.17.39:5555"]
        field => ["ZMQ_Response"]
        timeout => 1000
        retries => 10
    }

When I send the data to ZMQ, I am getting zmqtimeouts. To investigate, I have done the following :

1. Convert my server code to multi-threaded (ZeroMQ is fast, but just to make sure).
2. Stress test the server for about 10,000 request/sec (My traffic and use case is close to 2000 requests/sec).
3. Use tcpdump to figure out, network issues, but haven't found any realiable evidence there.