Logstash google pubsub output plugin

Elastic Stack Logstash
1

hello All,

I am trying to injest to google pubsub topic from logstash server. Below are the configuration
google_pubsub {
project_id => "xx"
topic => "xx"
json_key_file => "xx"
#Options for configuring the upload
message_count_threshold => 1000
delay_threshold_secs => 10
id => "GOOGLE-PUBSUB"
I am continuously receiving ":error=>"io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 9999594176ns" errors and data is not injested. Can someone plesae help me out. I am using logstash version 7. Using system default java.
Thanks

2

HI @Bala_Joshi, Welcome to the Elastic community.

As I tested with below conf

output {
  google_pubsub {
    project_id => "elastic-community"
    topic => "test-logstash"
    json_key_file => "my-key.json"
  }
}

It pushed successfully.

This is the issue of timeout while publishing the message.

As I can check here https://cloud.google.com/pubsub/docs/troubleshooting#publish-deadline-exceeded This is likely caused by a client side bottleneck, such as insufficient service CPUs, bad thread health, or network congestion.

You can try to tune message_count_threshold and delay_threshold_secs value and check what is the best fit.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.