ERROR: Unknown command 'Logstash'

Hi

I'm new to Logstash and I am following this blog (https://www.elastic.co/blog/logstash-jdbc-input-plugin) to connect postgresql to elasticsearch using logstash, but I am getting this error:

logstash-7.2.0/bin/logstash -f simple-out.conf Logstash startup
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/media/kevin/Kevin2017/work/psql-jdbc/logstash-7.2.0/logstash-core/lib/jars/jruby-complete-9.2.7.0.jar) to field java.io.FileDescriptor.fd
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Thread.exclusive is deprecated, use Thread::Mutex
ERROR: Unknown command 'Logstash'

See: 'bin/logstash --help'
[ERROR] 2019-06-26 23:32:17.130 [main] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

I am using:

Ubuntu 18.04
Postgres version: 10.8
Elasticsearch version: 7.1.1
Logstash: 7.2.0

Please Help.

logstash 1.5.3 is a very, very long time ago. Try

logstash-7.2.0/bin/logstash -f simple-out.conf
1 Like

Hi Badger, Thanks for your response, the error is now gone. but it come up with another, I'm not sure but I think it can't read the postgres jdbc driver?

logstash-7.2.0/bin/logstash -f simple-out.conf
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/media/kevin/Kevin2017/work/psql-jdbc/logstash-7.2.0/logstash-core/lib/jars/jruby-complete-9.2.7.0.jar) to field java.io.FileDescriptor.fd
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Thread.exclusive is deprecated, use Thread::Mutex
Sending Logstash logs to /media/kevin/Kevin2017/work/psql-jdbc/logstash-7.2.0/logs which is now configured via log4j2.properties
[2019-06-27T01:07:02,652][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-06-27T01:07:02,716][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.2.0"}
[2019-06-27T01:07:05,416][WARN ][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge] A gauge metric of an unknown type (org.jruby.RubyArray) has been create for key: cluster_uuids. This may result in invalid serialization.  It is recommended to log an issue to the responsible developer/development team.
[2019-06-27T01:07:05,422][INFO ][logstash.javapipeline    ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, :thread=>"#<Thread:0xc89fbb9 run>"}
[2019-06-27T01:07:07,674][INFO ][logstash.javapipeline    ] Pipeline started {"pipeline.id"=>"main"}
[2019-06-27T01:07:07,804][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-06-27T01:07:09,192][ERROR][logstash.inputs.jdbc     ] Failed to load /media/kevin/Kevin2017/work/psql-jdbc/postgresql-42.2.6.jar {:exception=>#<TypeError: failed to coerce jdk.internal.loader.ClassLoaders$AppClassLoader to java.net.URLClassLoader>}
[2019-06-27T01:07:09,201][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2019-06-27T01:07:09,250][ERROR][logstash.javapipeline    ] A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
Plugin: <LogStash::Inputs::Jdbc jdbc_user=>"postgres", statement=>"SELECT * from maincat", 
jdbc_driver_library=>"/media/kevin/Kevin2017/work/psql-jdbc/postgresql-42.2.6.jar", 
jdbc_connection_string=>"jdbc:postgresql://localhost:5432/kwheel", 
id=>"7df427291a77662b8c8485704940b8cf41c073e65835618e85f587fbcaa57dc0", jdbc_driver_class=>"org.postgresql.Driver", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_0b6fc7e0-d7b7-46b5-9488-2afbbb88a4a2", enable_metric=>true, charset=>"UTF-8">, jdbc_paging_enabled=>false, jdbc_page_size=>100000, jdbc_validate_connection=>false, jdbc_validation_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>"info", connection_retry_attempts=>1, connection_retry_attempts_wait_time=>0.5, parameters=>{"sql_last_value"=>1970-01-01 00:00:00 UTC}, last_run_metadata_path=>"/home/kevin/.logstash_jdbc_last_run", use_column_value=>false, tracking_column_type=>"numeric", clean_run=>false, record_last_run=>true, lowercase_column_names=>true>
Error: org.postgresql.Driver not loaded. Are you sure you've included the correct jdbc driver in :jdbc_driver_library?
Exception: LogStash::ConfigurationError
 Stack: /media/kevin/Kevin2017/work/psql-jdbc/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.13/lib/logstash/plugin_mixins/jdbc/jdbc.rb:163:in `open_jdbc_connection'
/media/kevin/Kevin2017/work/psql-jdbc/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.13/lib/logstash/plugin_mixins/jdbc/jdbc.rb:221:in `execute_statement'
/media/kevin/Kevin2017/work/psql-jdbc/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.13/lib/logstash/inputs/jdbc.rb:277:in `execute_query'
/media/kevin/Kevin2017/work/psql-jdbc/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.13/lib/logstash/inputs/jdbc.rb:263:in `run'
/media/kevin/Kevin2017/work/psql-jdbc/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:309:in `inputworker'
/media/kevin/Kevin2017/work/psql-jdbc/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:302:in `block in start_input'
[2019-06-27T01:07:10,260][ERROR][logstash.inputs.jdbc     ] Failed to load /media/kevin/Kevin2017/work/psql-jdbc/postgresql-42.2.6.jar {:exception=>#<TypeError: failed to coerce jdk.internal.loader.ClassLoaders$AppClassLoader to java.net.URLClassLoader>}
[2019-06-27T01:07:10,264][ERROR][logstash.javapipeline    ] A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
 Plugin: <LogStash::Inputs::Jdbc jdbc_user=>"postgres", statement=>"SELECT * from maincat", 
jdbc_driver_library=>"/media/kevin/Kevin2017/work/psql-jdbc/postgresql-42.2.6.jar", jdbc_connection_string=>"jdbc:postgresql://localhost:5432/kwheel", id=>"7df427291a77662b8c8485704940b8cf41c073e65835618e85f587fbcaa57dc0", jdbc_driver_class=>"org.postgresql.Driver", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_0b6fc7e0-d7b7-46b5-9488-2afbbb88a4a2", enable_metric=>true, charset=>"UTF-8">, jdbc_paging_enabled=>false, jdbc_page_size=>100000, jdbc_validate_connection=>false, jdbc_validation_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>"info", connection_retry_attempts=>1, connection_retry_attempts_wait_time=>0.5, parameters=>{"sql_last_value"=>1970-01-01 00:00:00 UTC}, last_run_metadata_path=>"/home/kevin/.logstash_jdbc_last_run", use_column_value=>false, tracking_column_type=>"numeric", clean_run=>false, record_last_run=>true, lowercase_column_names=>true>
Error: org.postgresql.Driver not loaded. Are you sure you've included the correct jdbc driver in :jdbc_driver_library?
Exception: LogStash::ConfigurationError
1 Like

Known issue. A workaround is mentioned here. Read down to the end of the thread for additional issues you might face.

2 Likes

Hi Badger!

Thanks for the link, it helps me a lot! after a few trial and errors I already got it working! :slight_smile:

Hi,
I was looking for a solution for my warning and ended up here ... in these logs I see the same warning, which came out without changing anything:

Can you help me?

I started seeing that warning after upgrading to 7.2.0. It is a known issue.

1 Like

I noticed another strange warning: Thread.exclusive is deprecated, use Thread::Mutex.
Is this a recognized warning too? @Badger

Nobody has opened an issue for it yet, but I also get that starting with 7.2.0.

1 Like

Thanks!!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.