Couldn't find any input plugin named 'google_pubsub'. Are you sure this is correct?

Hello,

I noticed Kibana is no longer having Google cloud Stackdriver logs. So I started my troubleshooting with logstash and debug the .conf file, here's what I got:

[2021-08-02T20:08:44,540][DEBUG][logstash.plugins.registry] Problems loading the plugin with {:type=>"input", :name=>"google_pubsub"}
[2021-08-02T20:08:44,566][FATAL][logstash.runner ] The given configuration is invalid. Reason: Unable to configure plugins: (PluginLoadingError) Couldn't find any input plugin named 'google_pubsub'. Are you sure this is correct? Trying to load the google_pubsub input plugin resulted in this error: no such file to load -- logstash/inputs/google_pubsub

Here's the google.conf file config:

input
{
google_pubsub {
project_id => ""
topic => ""
subscription => ""
json_key_file => "/opt/pubsubkey/pubsubkey.json"
include_metadata => true
codec => "json"
type => "gcp"
}

heartbeat {
    interval => 10
    type => "heartbeat"
}

}
filter {
if [type] != "heartbeat" {
mutate {
add_field => { "messageId" => "%{[@metadata][pubsub_message][messageId]}" }
}
}
}
output
{
if [type] == "gcp" {
elasticsearch {
hosts => [ "localhost:9200" ]
index => "logstash-events-%{+YYYY.MM.dd}"
}
}

}

Is there anything which is changed about plugin but I have not even upgraded Elastic. It's on 7.9.2 still? Please help me to find out more on this.

Have you installed the pubsub plugin?

The issue has been resolved by installing the google-pubsub plugin. Unsure why it wasn't there when everything was working perfectly fine until last week.

Any guidance on if this has happened due to some misconfiguration or something else?

1 Like

wild guess: Upgrade. If you installed via rpm/deb package, logstash might have been upgraded, which does not install that plugin automatically with the new version.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.