Hello,
I noticed Kibana is no longer having Google cloud Stackdriver logs. So I started my troubleshooting with logstash and debug the .conf file, here's what I got:
[2021-08-02T20:08:44,540][DEBUG][logstash.plugins.registry] Problems loading the plugin with {:type=>"input", :name=>"google_pubsub"}
[2021-08-02T20:08:44,566][FATAL][logstash.runner ] The given configuration is invalid. Reason: Unable to configure plugins: (PluginLoadingError) Couldn't find any input plugin named 'google_pubsub'. Are you sure this is correct? Trying to load the google_pubsub input plugin resulted in this error: no such file to load -- logstash/inputs/google_pubsub
Here's the google.conf file config:
input
{
google_pubsub {
project_id => ""
topic => ""
subscription => ""
json_key_file => "/opt/pubsubkey/pubsubkey.json"
include_metadata => true
codec => "json"
type => "gcp"
}
heartbeat {
interval => 10
type => "heartbeat"
}
}
filter {
if [type] != "heartbeat" {
mutate {
add_field => { "messageId" => "%{[@metadata][pubsub_message][messageId]}" }
}
}
}
output
{
if [type] == "gcp" {
elasticsearch {
hosts => [ "localhost:9200" ]
index => "logstash-events-%{+YYYY.MM.dd}"
}
}
}
Is there anything which is changed about plugin but I have not even upgraded Elastic. It's on 7.9.2 still? Please help me to find out more on this.