I installed logstash on a debian OS and then installed the Google Pub Sub plugin . Simple pipeline test works fine, but using Google pub/sub as input results in below error.
last line of stacktrace:
[ERROR] 2018-05-15 20:48:06.544 [LogStash::Runner] Logstash - java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;J)V
sudo bin/logstash --version
logstash 6.2.4
/usr/share/logstash$ sudo bin/logstash -f gcp-pubsub-logstash.conf --config.reload.automatic
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2018-05-15 20:48:02.415 [main] scaffold - Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[INFO ] 2018-05-15 20:48:02.443 [main] scaffold - Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[WARN ] 2018-05-15 20:48:03.490 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2018-05-15 20:48:03.824 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.2.4"}
[INFO ] 2018-05-15 20:48:04.243 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2018-05-15 20:48:05.908 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2018-05-15 20:48:06.085 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x2ef216d6@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:247 run>"}
[INFO ] 2018-05-15 20:48:06.140 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Pipelines running {:count=>1, :pipelines=>["main"]}
[INFO ] 2018-05-15 20:48:06.515 [[main]-pipeline-manager] pipeline - Pipeline has terminated {:pipeline_id=>"main", :thread=>"#<Thread:0x2ef216d6@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:247 run>"}
[ERROR] 2018-05-15 20:48:06.544 [LogStash::Runner] Logstash - java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;J)V
cat gcp-pubsub-logstash.conf
The # character at the beginning of a line indicates a comment. Use
comments to describe your configuration.
input {
google_pubsub {
# Your GCP project id (name)
project_id => "pinkesh-autocomplete-202716"
# The topic name below is currently hard-coded in the plugin. You
# must first create this topic by hand and ensure you are exporting
# logging to this pubsub topic.
topic => "product-ingestion-topic"
# The subscription name is customizeable. The plugin will attempt to
# create the subscription (but use the hard-coded topic name above).
subscription => "mysubs"
# If you are running logstash within GCE, it will use
# Application Default Credentials and use GCE's metadata
# service to fetch tokens. However, if you are running logstash
# outside of GCE, you will need to specify the service account's
# JSON key file below.
#json_key_file => "/home/erjohnso/pkey.json"
}
}
filter {
mutate {
add_field => { "messageId" => "%{[@metadata][pubsub_message][messageId]}" }
}
}
output { stdout { codec => rubydebug } }
/usr/share/logstash$ java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-8u171-b11-1~deb9u1-b11)
OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode)
I had to create a symlink for Logstash to install correctly since it is by default using /usr/bin/java
ls -l /usr/bin/java
lrwxrwxrwx 1 root root 42 May 15 19:58 /usr/bin/java -> /usr/lib/jvm/java-8-openjdk-amd64/bin/java
echo $JAVA_HOME
/usr/lib/jvm/java-8-openjdk-amd64
echo $PATH
/usr/lib/jvm/java-8-openjdk-amd64/bin:/opt/bitnami/elasticsearch/bin:/opt/bitnami/apache2/bin:/opt/bitnami/common/bin:/opt/bitnami/java/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games