Hello,
I am having trouble giving input to the elasticearch from data of MySQL via logstash.
The following is my simple config file:
input {
jdbc {
clean_run => true
jdbc_driver_library => "mysql-connector-java-5.1.49-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:3306/honeypot"
jdbc_user => "root"
jdbc_password => "1234"
statement => "SELECT * from honLogs"
}
}
output{
elasticsearch {
hosts => ["localhost:9200"]
index => "honLogs"
}
stdout { codec => rubydebug }
}
I am getting this error when trying to upload to Kibana via logstash:
C:\Elastic_stack\logstash-7.8.1>logstash -f config/logstash_mysql_1.conf
Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option UseConcMarkSweepGC; support was removed in 14.0
Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option CMSInitiatingOccupancyFraction; support was removed in 14.0
Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option UseCMSInitiatingOccupancyOnly; support was removed in 14.0
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/C:/Elastic_stack/logstash-7.8.1/logstash-core/lib/jars/jruby-complete-9.2.11.1.jar) to field java.io.Console.cs
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Sending Logstash logs to C:/Elastic_stack/logstash-7.8.1/logs which is now configured via log4j2.properties
[2021-01-11T16:40:44,644][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-01-11T16:40:44,733][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.8.1", "jruby.version"=>"jruby 9.2.11.1 (2.5.7) 2020-03-25 b1f55b1a40 Java HotSpot(TM) 64-Bit Server VM 14.0.2+12-46 on 14.0.2+12-46 +indy +jit [mswin32-x86_64]"}
[2021-01-11T16:40:46,066][INFO ][org.reflections.Reflections] Reflections took 37 ms to scan 1 urls, producing 21 keys and 41 values
[2021-01-11T16:40:47,048][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>, :added=>[http://localhost:9200/]}}
[2021-01-11T16:40:47,200][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2021-01-11T16:40:47,240][INFO ][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
[2021-01-11T16:40:47,244][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the type event field won't be used to determine the document _type {:es_version=>7}
[2021-01-11T16:43:45,772][ERROR][logstash.javapipeline ][main][0de6c206ed3e0338a2d4327f3f018becd4ad4987b50eabb1f550d6d5a664c387] A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
Plugin: <LogStash::Inputs::Jdbc jdbc_user=>"root", jdbc_password=>, statement=>"SELECT * from honLogs", clean_run=>true, jdbc_driver_library=>"mysql-connector-java-5.1.49-bin.jar", jdbc_connection_string=>"jdbc:mysql://localhost:3306/honeypot", id=>"0de6c206ed3e0338a2d4327f3f018becd4ad4987b50eabb1f550d6d5a664c387", jdbc_driver_class=>"com.mysql.jdbc.Driver", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_c8b47113-95c9-4912-82bb-fb9645950746", enable_metric=>true, charset=>"UTF-8">, jdbc_paging_enabled=>false, jdbc_page_size=>100000, jdbc_validate_connection=>false, jdbc_validation_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>"info", connection_retry_attempts=>1, connection_retry_attempts_wait_time=>0.5, plugin_timezone=>"utc", last_run_metadata_path=>"C:\Users\itlab/.logstash_jdbc_last_run", use_column_value=>false, tracking_column_type=>"numeric", record_last_run=>true, lowercase_column_names=>true, use_prepared_statements=>false>
Error: unable to load mysql-connector-java-5.1.49-bin.jar from :jdbc_driver_library, file not readable (please check user and group permissions for the path)
Exception: LogStash::PluginLoadingError
Stack: C:/Elastic_stack/logstash-7.8.1/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.5/lib/logstash/plugin_mixins/jdbc/common.rb:47:in block in load_driver_jars' org/jruby/RubyArray.java:1809:in each'
C:/Elastic_stack/logstash-7.8.1/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.5/lib/logstash/plugin_mixins/jdbc/common.rb:42:in load_driver_jars' C:/Elastic_stack/logstash-7.8.1/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.5/lib/logstash/plugin_mixins/jdbc/common.rb:25:in load_driver'
C:/Elastic_stack/logstash-7.8.1/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.5/lib/logstash/inputs/jdbc.rb:275:in run' C:/Elastic_stack/logstash-7.8.1/logstash-core/lib/logstash/java_pipeline.rb:374:in inputworker'
C:/Elastic_stack/logstash-7.8.1/logstash-core/lib/logstash/java_pipeline.rb:365:in `block in start_input'
Let me know what's the mistake how it can be rectified
Thank you in advance.