Couldn't find any input plugin named 'google_pubsub'. Are you sure this is correct?

[devashishsingh]

Hello,

I noticed Kibana is no longer having Google cloud Stackdriver logs. So I started my troubleshooting with logstash and debug the .conf file, here's what I got:

[2021-08-02T20:08:44,540][DEBUG][logstash.plugins.registry] Problems loading the plugin with {:type=>"input", :name=>"google_pubsub"}
[2021-08-02T20:08:44,566][FATAL][logstash.runner ] The given configuration is invalid. Reason: Unable to configure plugins: (PluginLoadingError) Couldn't find any input plugin named 'google_pubsub'. Are you sure this is correct? Trying to load the google_pubsub input plugin resulted in this error: no such file to load -- logstash/inputs/google_pubsub

Here's the google.conf file config:

input
{
google_pubsub {
project_id => ""
topic => ""
subscription => ""
json_key_file => "/opt/pubsubkey/pubsubkey.json"
include_metadata => true
codec => "json"
type => "gcp"
}

heartbeat {
    interval => 10
    type => "heartbeat"
}

}
filter {
if [type] != "heartbeat" {
mutate {
add_field => { "messageId" => "%{[@metadata][pubsub_message][messageId]}" }
}
}
}
output
{
if [type] == "gcp" {
elasticsearch {
hosts => [ "localhost:9200" ]
index => "logstash-events-%{+YYYY.MM.dd}"
}
}

}

Is there anything which is changed about plugin but I have not even upgraded Elastic. It's on 7.9.2 still? Please help me to find out more on this.

[spinscale (Alexander Reelsen)]

Have you installed the pubsub plugin?

[devashishsingh]

The issue has been resolved by installing the google-pubsub plugin. Unsure why it wasn't there when everything was working perfectly fine until last week.

Any guidance on if this has happened due to some misconfiguration or something else?

[spinscale (Alexander Reelsen)]

wild guess: Upgrade. If you installed via rpm/deb package, logstash might have been upgraded, which does not install that plugin automatically with the new version.