Not able to create index using logstash for my ES cluster

Hello,

I am using jdbc input plugin with logstash to fetch data from database and create index in ES.
But when i am trying to run config file it is not getting started.
I am having 3 node cluster with one master node and 2 data nodes.

Do i need to write all the hosts name in my logstash config file?

My config file look like (on master):

input {
jdbc {
jdbc_driver_library => "E:\oracle\product\12.2.0\client_1\jdbc\ojdbc8.jar"
jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
jdbc_connection_string => "db connection string"
jdbc_user => "username"
jdbc_password => "password"
statement => "sql statement"
last_run_metadata_path => "E:/usr/share/logstash/last_run_metadata/Master.txt"
use_column_value => true
tracking_column => "version"
schedule=> "0 * * * *"
}
}
output{
elasticsearch {
index => "new_database"
}
}

Regards,
Priyanka

Hi @pyerunka,

If you from the machine that has the above Logstash config can get to http://127.0.0.1:9200
then you should be fine. Try curl or telnet or something like that. Looks like you are using Windows. Not sure what tools you have available...

More info https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-hosts

Logstash logs are usually quite good at telling what is wrong. Can you see anything there?

I'm not 100% is it is mandatory but Logstash configs usually have a filter section. You can just add

filter {}

but the logs should show if that is an issue or not...

Hello @A_B,

I am getting below error:

E:\logstash-7.4.2\bin>logstash -f Master.conf
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.runtime.encoding.EncodingService(file:/E:/logstash-7.4.2/logstash-core/lib/jars/jruby-complete-9.2.8.0.jar) to
field java.io.Console.cs
WARNING: Please consider reporting this to the maintainers of org.jruby.runtime.
encoding.EncodingService
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflect
ive access operations
WARNING: All illegal access operations will be denied in a future release
Thread.exclusive is deprecated, use Thread::Mutex
Sending Logstash logs to E:/logstash-7.4.2/logs which is now configured via log4
j2.properties
[2020-01-27T16:39:48,440][WARN ][logstash.config.source.multilocal] Ignoring the
'pipelines.yml' file because modules or command line options are specified
[2020-01-27T16:39:48,552][INFO ][logstash.runner ] Starting Logstash {"
logstash.version"=>"7.4.2"}
[2020-01-27T16:39:51,665][INFO ][org.reflections.Reflections] Reflections took 4
8 ms to scan 1 urls, producing 20 keys and 40 values
[2020-01-27T16:39:52,017][ERROR][logstash.outputs.elasticsearch] Unknown setting
'host' for elasticsearch
[2020-01-27T16:39:52,029][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (
ConfigurationError) Something is wrong with your configuration.", :backtrace=>["
org.logstash.config.ir.CompiledPipeline.(CompiledPipeline.java:100)", "org
.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:60)"
, "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(Java
BasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.meth
ods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRR
untimeHelpers.instanceSuper(IRRuntimeHelpers.java:1156)", "org.jruby.ir.runtime.
IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1143)", "org.jruby
.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "E
3a.logstash_minus_7_dot_4_dot_2.logstash_minus_core.lib.logstash.java_pipeline
.RUBY$method$initialize$0(E:/logstash-7.4.2/logstash-core/lib/logstash/java_pipe
line.rb:26)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(Compiled
IRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(M
ixedModeIRMethod.java:90)", "org.jruby.runtime.callsite.CachingCallSite.cacheAnd
Call(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCallSite.cal
l(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.java:915
)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInsta
nce.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "E_3a_
.logstash_minus_7_dot_4_dot_2.logstash_minus_core.lib.logstash.pipeline_action.c
reate.RUBY$method$execute$0(E:/logstash-7.4.2/logstash-core/lib/logstash/pipelin
e_action/create.rb:36)", "E_3a_.logstash_minus_7_dot_4_dot_2.logstash_minus_core
.lib.logstash.pipeline_action.create.RUBY$method$execute$0$VARARGS(E:/logsta
sh-7.4.2/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jruby.inte
rnal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jrub
y.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "
org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "E_3a_.logstash_mi
nus_7_dot_4_dot_2.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_sta
te$2(E:/logstash-7.4.2/logstash-core/lib/logstash/agent.rb:326)", "org.jruby.run
time.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)", "org.jruby.r
untime.IRBlockBody.call(IRBlockBody.java:77)", "org.jruby.runtime.Block.call(Blo
ck.java:129)", "org.jruby.RubyProc.call(RubyProc.java:295)", "org.jruby.RubyProc
.call(RubyProc.java:274)", "org.jruby.RubyProc.call(RubyProc.java:270)", "org.jr
uby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.base/java.l
ang.Thread.run(Thread.java:834)"]}
warning: thread "Converge PipelineAction::Create" terminated with exceptio
n (report_on_exception is true):
LogStash::Error: Don't know how to handle Java::JavaLang::IllegalStateException for PipelineAction::Create<main>
create at org/logstash/execution/ConvergeResultExt.java:109
add at org/logstash/execution/ConvergeResultExt.java:37
converge_state at E:/logstash-7.4.2/logstash-core/lib/logstash/agent.rb:339
[2020-01-27T16:39:52,037][ERROR][logstash.agent ] An exception happene
d when converging configuration {:exception=>LogStash::Error, :message=>"Don't k
now how to handle Java::JavaLang::IllegalStateException for PipelineAction::C reate<main>", :backtrace=>["org/logstash/execution/ConvergeResultExt.java:109:i
n create'", "org/logstash/execution/ConvergeResultExt.java:37:in add'", "E:/lo
gstash-7.4.2/logstash-core/lib/logstash/agent.rb:339:in block in converge_state '"]} [2020-01-27T16:39:52,213][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600} [2020-01-27T16:39:52,297][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle Java::JavaLang::
IllegalStateExceptionforPipelineAction::Create>, :backtrace=>["org/lo gstash/execution/ConvergeResultExt.java:109:in create'", "org/logstash/executio
n/ConvergeResultExt.java:37:in add'", "E:/logstash-7.4.2/logstash-core/lib/logs tash/agent.rb:339:in block in converge_state'"]}
[2020-01-27T16:39:53,933][ERROR][org.logstash.Logstash ] java.lang.IllegalSta
teException: Logstash stopped processing because of an error: (SystemExit) exit

Regards,
Priyanka

In the configuration of your elasticsearch output change 'host =>' to 'hosts =>'

Hello @Badger,

I have tried with hosts, but still it is showing me below error:

E:\logstash-7.4.2\bin>logstash -f Master.conf
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was depreca
ted in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.runtime.encoding.EncodingService
(file:/E:/logstash-7.4.2/logstash-core/lib/jars/jruby-complete-9.2.8.0.jar) to
field java.io.Console.cs
WARNING: Please consider reporting this to the maintainers of org.jruby.runtime.
encoding.EncodingService
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflect
ive access operations
WARNING: All illegal access operations will be denied in a future release
Thread.exclusive is deprecated, use Thread::Mutex
Sending Logstash logs to E:/logstash-7.4.2/logs which is now configured via log4
j2.properties
[2020-01-27T18:30:07,707][WARN ][logstash.config.source.multilocal] Ignoring the
'pipelines.yml' file because modules or command line options are specified
[2020-01-27T18:30:07,719][INFO ][logstash.runner ] Starting Logstash {"
logstash.version"=>"7.4.2"}
[2020-01-27T18:30:09,299][INFO ][org.reflections.Reflections] Reflections took 3
6 ms to scan 1 urls, producing 20 keys and 40 values
[2020-01-27T18:30:09,603][ERROR][logstash.agent ] Failed to execute ac
tion {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"J
ava::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: I
llegal character in opaque part at index 5: http:\\localhosts:9200", :backt
race=>["org.logstash.config.ir.CompiledPipeline.(CompiledPipeline.java:100
)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.j
ava:60)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.c
all(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runt
ime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.run
time.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1156)", "org.jruby.ir.
runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1143)", "o
rg.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:
39)", "E_3a_.logstash_minus_7_dot_4_dot_2.logstash_minus_core.lib.logstash.java_
pipeline.RUBY$method$initialize$0(E:/logstash-7.4.2/logstash-core/lib/logstash/j
ava_pipeline.rb:26)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(
CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMetho
d.call(MixedModeIRMethod.java:90)", "org.jruby.runtime.callsite.CachingCallSite.
cacheAndCall(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCall
Site.call(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.
java:915)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$
newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)"
, "E_3a_.logstash_minus_7_dot_4_dot_2.logstash_minus_core.lib.logstash.pipeline_
action.create.RUBY$method$execute$0(E:/logstash-7.4.2/logstash-core/lib/logstash
/pipeline_action/create.rb:36)", "E_3a_.logstash_minus_7_dot_4_dot_2.logstash_mi
nus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$VARARGS(E
:/logstash-7.4.2/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jr
uby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java
:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "E_3a_.log
stash_minus_7_dot_4_dot_2.logstash_minus_core.lib.logstash.agent.RUBY$block$conv
erge_state$2(E:/logstash-7.4.2/logstash-core/lib/logstash/agent.rb:326)", "org.j
ruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)", "org
.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)", "org.jruby.runtime.Block.
call(Block.java:129)", "org.jruby.RubyProc.call(RubyProc.java:295)", "org.jruby.
RubyProc.call(RubyProc.java:274)", "org.jruby.RubyProc.call(RubyProc.java:270)",
"org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.bas
e/java.lang.Thread.run(Thread.java:834)"]}
warning: thread "Converge PipelineAction::Create" terminated with exceptio
n (report_on_exception is true):
LogStash::Error: Don't know how to handle Java::JavaLang::IllegalStateException for PipelineAction::Create<main>
create at org/logstash/execution/ConvergeResultExt.java:109
add at org/logstash/execution/ConvergeResultExt.java:37
converge_state at E:/logstash-7.4.2/logstash-core/lib/logstash/agent.rb:339
[2020-01-27T18:30:09,611][ERROR][logstash.agent ] An exception happene
d when converging configuration {:exception=>LogStash::Error, :message=>"Don't k
now how to handle Java::JavaLang::IllegalStateException for PipelineAction::C reate<main>", :backtrace=>["org/logstash/execution/ConvergeResultExt.java:109:i
n create'", "org/logstash/execution/ConvergeResultExt.java:37:in add'", "E:/lo
gstash-7.4.2/logstash-core/lib/logstash/agent.rb:339:in block in converge_state '"]} [2020-01-27T18:30:09,799][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600} [2020-01-27T18:30:09,879][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle Java::JavaLang::
IllegalStateExceptionforPipelineAction::Create>, :backtrace=>["org/lo gstash/execution/ConvergeResultExt.java:109:in create'", "org/logstash/executio
n/ConvergeResultExt.java:37:in add'", "E:/logstash-7.4.2/logstash-core/lib/logs tash/agent.rb:339:in block in converge_state'"]}
[2020-01-27T18:30:11,491][ERROR][org.logstash.Logstash ] java.lang.IllegalSta
teException: Logstash stopped processing because of an error: (SystemExit) exit

Regards,
Priyanka

What does you configuration look like?

Hello @Badger,

I have mentioned my config file in above updates.

Regards,
Priyanka

That is definitely a problem. It should be hosts not host. Are you still using the same config as you posted originally? The config would be easier to read of it was formatted correctly. Could you try changing from Blockquote to Preformatted text :slight_smile:

Hello @A_B,

I am using hosts only.

Please find my updated config file ;

input {
jdbc {
jdbc_driver_library => "E:/oracle/product/12.2.0/client_1/jdbc/ojdbc8.jar"
jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
jdbc_connection_string => "jdbcconnectionstring"
jdbc_user => "username"
jdbc_password => "pwd"
statement => "select * from db_table"
last_run_metadata_path => "E:/usr/share/logstash/last_run_metadata/Master.txt"
}
}
output{
elasticsearch {
hosts => ["http:\localhost:9200"]
index => "new_database"
}
}

And you have also mentioned to change Blockquote to Preformatted text.
What needs to be done actually? Can you please explain.

Regards,
Priyanka

That is not a valid url. It should start with ‘http://‘.

That should be "http://localhost:9200"

From the text formatting tools

This is Blockquote
This line was originally indented but is not shown as such

This is Preformatted text
    This line was originally indented and still is

Hello @A_B, @Badger, @Christian_Dahlqvist

I have changed the above mentioned changes and try to run logstash config file. I am now getting below error:

  Error: Unable to find driver class via URLClassLoader in given driver jars: Ja
va::oracle.jdbc.driver.OracleDriver and Java::oracle.jdbc.driver.OracleDriver
  Exception: LogStash::PluginLoadingError
  Stack: E:/logstash-7.4.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.
3.16/lib/logstash/plugin_mixins/jdbc/jdbc.rb:181:in `open_jdbc_connection'
E:/logstash-7.4.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.16/lib/
logstash/plugin_mixins/jdbc/jdbc.rb:253:in `execute_statement'
E:/logstash-7.4.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.16/lib/
logstash/inputs/jdbc.rb:309:in `execute_query'
E:/logstash-7.4.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.16/lib/
logstash/inputs/jdbc.rb:281:in `run'
E:/logstash-7.4.2/logstash-core/lib/logstash/java_pipeline.rb:314:in `inputworke
r'
E:/logstash-7.4.2/logstash-core/lib/logstash/java_pipeline.rb:306:in `block in s
tart_input'
[2020-01-28T08:52:29,052][ERROR][logstash.javapipeline    ][main] A plugin had a
n unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::Jdbc jdbc_user=>"username", last_run_metadata_path=>"
E:/usr/share/logstash/last_run_metadata/Master.txt", jdbc_password=><password>,
statement=>"select * from db_table", jdbc_driver_library=>"E:/oracle/product/12.2.0/client_1/jdbc/ojd
bc8.jar", jdbc_connection_string=>"jdbc:oracle:thin:@//nl0123vca0020.amsterdam-n
l0123.slb.com:1521/BINT1", id=>"dc1fda780d75da25bbdb7f57679be5e48e3d4f8713c55718
786795898d0c38b1", jdbc_driver_class=>"Java::oracle.jdbc.driver.OracleDriver", e
nable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_a1e70242-a2b8-407
3-8136-28478a49d5f1", enable_metric=>true, charset=>"UTF-8">, jdbc_paging_enable
d=>false, jdbc_page_size=>100000, jdbc_validate_connection=>false, jdbc_validati
on_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>"info", connection_retry_
attempts=>1, connection_retry_attempts_wait_time=>0.5, plugin_timezone=>"utc", u
se_column_value=>false, tracking_column_type=>"numeric", clean_run=>false, recor
d_last_run=>true, lowercase_column_names=>true, use_prepared_statements=>false>
  Error: Unable to find driver class via URLClassLoader in given driver jars: Ja
va::oracle.jdbc.driver.OracleDriver and Java::oracle.jdbc.driver.OracleDriver
  Exception: LogStash::PluginLoadingError
  Stack: E:/logstash-7.4.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.
3.16/lib/logstash/plugin_mixins/jdbc/jdbc.rb:181:in `open_jdbc_connection'
E:/logstash-7.4.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.16/lib/
logstash/plugin_mixins/jdbc/jdbc.rb:253:in `execute_statement'
E:/logstash-7.4.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.16/lib/
logstash/inputs/jdbc.rb:309:in `execute_query'
E:/logstash-7.4.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.16/lib/
logstash/inputs/jdbc.rb:281:in `run'
E:/logstash-7.4.2/logstash-core/lib/logstash/java_pipeline.rb:314:in `inputworke
r'
E:/logstash-7.4.2/logstash-core/lib/logstash/java_pipeline.rb:306:in `block in s
tart_input' 

Regards,
Priyanka

@pyerunka , I see that you are using Logstash version 7.4.2 (Windows) with logstash-input-jdbc plugin version 4.3.16. You are encountering the following error:

Error: Unable to find driver class via URLClassLoader in given driver jars: Ja
va::oracle.jdbc.driver.OracleDriver and Java::oracle.jdbc.driver.OracleDriver
  Exception: LogStash::PluginLoadingError

This error could be related to a known issue with the JDBC driver loading operation. This issue got resolved in the JDBC input plugin version 4.3.18.

There are two options you can explore in order to update the JDBC input plugin to the latest version:

Could you try this and let us know if this solves the issue?

Thank you.

Hello @ropc,

I have tried updating plugin using bin\logstash-plugin update. It is giving me below error:

> 2020-01-28T14:55:40.089Z [main] WARN FilenoUtil : Native subprocess control requ
> ires open access to sun.nio.ch
> Pass '--add-opens java.base/sun.nio.ch=org.jruby.dist' or '=org.jruby.core' to e
> nable.
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file
> :/E:/logstash-7.4.2/vendor/jruby/lib/ruby/stdlib/jopenssl.jar) to constructor ja
> va.security.cert.CertificateFactory(java.security.cert.CertificateFactorySpi,jav
> a.security.Provider,java.lang.String)
> WARNING: Please consider reporting this to the maintainers of org.jruby.ext.open
> ssl.SecurityHelper
> WARNING: Use --illegal-access=warn to enable warnings of further illegal reflect
> ive access operations
> WARNING: All illegal access operations will be denied in a future release
> Updating logstash-codec-avro, logstash-codec-cef, logstash-codec-collectd, logst
> ash-codec-dots, logstash-codec-edn, logstash-codec-edn_lines, logstash-codec-es_
> bulk, logstash-codec-fluent, logstash-codec-graphite, logstash-codec-json, logst
> ash-codec-json_lines, logstash-codec-line, logstash-codec-msgpack, logstash-code
> c-multiline, logstash-codec-netflow, logstash-codec-plain, logstash-codec-rubyde
> bug, logstash-filter-aggregate, logstash-filter-anonymize, logstash-filter-cidr,
>  logstash-filter-clone, logstash-filter-csv, logstash-filter-date, logstash-filt
> er-de_dot, logstash-filter-dissect, logstash-filter-dns, logstash-filter-drop, l
> ogstash-filter-elasticsearch, logstash-filter-fingerprint, logstash-filter-geoip
> , logstash-filter-grok, logstash-filter-http, logstash-filter-jdbc_static, logst
> ash-filter-jdbc_streaming, logstash-filter-json, logstash-filter-kv, logstash-fi
> lter-memcached, logstash-filter-metrics, logstash-filter-mutate, logstash-filter
> -prune, logstash-filter-ruby, logstash-filter-sleep, logstash-filter-split, logs
> tash-filter-syslog_pri, logstash-filter-throttle, logstash-filter-translate, log
> stash-filter-truncate, logstash-filter-urldecode, logstash-filter-useragent, log
> stash-filter-uuid, logstash-filter-xml, logstash-input-azure_event_hubs, logstas
> h-input-beats, logstash-input-couchdb_changes, logstash-input-dead_letter_queue,
>  logstash-input-elasticsearch, logstash-input-exec, logstash-input-file, logstas
> h-input-ganglia, logstash-input-gelf, logstash-input-generator, logstash-input-g
> raphite, logstash-input-heartbeat, logstash-input-http, logstash-input-http_poll
> er, logstash-input-imap, logstash-input-jdbc, logstash-input-jms, logstash-input
> -kafka, logstash-input-pipe, logstash-input-rabbitmq, logstash-input-redis, logs
> tash-input-s3, logstash-input-snmp, logstash-input-snmptrap, logstash-input-sqs,
>  logstash-input-stdin, logstash-input-syslog, logstash-input-tcp, logstash-input
> -twitter, logstash-input-udp, logstash-input-unix, logstash-output-cloudwatch, l
> ogstash-output-csv, logstash-output-elastic_app_search, logstash-output-elastics
> earch, logstash-output-email, logstash-output-file, logstash-output-graphite, lo
> gstash-output-http, logstash-output-kafka, logstash-output-lumberjack, logstash-
> output-nagios, logstash-output-null, logstash-output-pipe, logstash-output-rabbi
> tmq, logstash-output-redis, logstash-output-s3, logstash-output-sns, logstash-ou
> tput-sqs, logstash-output-stdout, logstash-output-tcp, logstash-output-udp, logs
> tash-output-webhdfs
> Error Bundler::HTTPError, retrying 1/10
> Could not fetch specs from https://rubygems.org/
> Error Bundler::HTTPError, retrying 2/10
> Could not fetch specs from https://rubygems.org/
> Error Bundler::HTTPError, retrying 3/10
> Could not fetch specs from https://rubygems.org/
> Error Bundler::HTTPError, retrying 4/10
> Could not fetch specs from https://rubygems.org/
> Error Bundler::HTTPError, retrying 5/10
> Could not fetch specs from https://rubygems.org/
> Error Bundler::HTTPError, retrying 6/10
> Could not fetch specs from https://rubygems.org/
> Error Bundler::HTTPError, retrying 7/10
> Could not fetch specs from https://rubygems.org/
> Error Bundler::HTTPError, retrying 8/10
> Could not fetch specs from https://rubygems.org/
> Error Bundler::HTTPError, retrying 9/10
> Could not fetch specs from https://rubygems.org/
> Error Bundler::HTTPError, retrying 10/10
> Could not fetch specs from https://rubygems.org/
> Too many retries, aborting, caused by Bundler::HTTPError
> ERROR: Updated Aborted, message: Could not fetch specs from https://rubygems.org/

And is this ok, if i use logstash version 7.5.2 with elasticsearch, kibana 7.4.2?

Regards,
Priyanka

  1. @pyerunka - The following error message seems to indicate some connectivity problem:
> Error Bundler::HTTPError, retrying 1/10
> Could not fetch specs from https://rubygems.org/
...
> Too many retries, aborting, caused by Bundler::HTTPError
> ERROR: Updated Aborted, message: Could not fetch specs from https://rubygems.org/

To update the Logstash plugins, you need connectivity to the Internet. Can you check your network and ensure that the host running Logstash can connect to the Internet? If there are network restrictions in your environment, you could consider using an offline plugin pack.

  1. If you check our product compatibility matrix, Logstash 7.5.x (i.e Elasticsearch output compatibility) can work with Elasticsearch / Kibana 7.4.x. There should not be any problems to have such environment. However, as a general rule, we recommend to have the same versions of the components across the stack.

I hope that helps.

Hello @ropc,

Thanks for your help !!!!.
My logstash issue has been resolved now. I am able to connect to sql database.
Updating logstash plugin not worked on my server. My server do not have any internet connectivity. May be because of that it has not worked.
I have created index using mentioned version of logstash 7.5.2.

Again thanks for your quick help!!!

Regards,
Priyanka

@pyerunka You're most welcome :slightly_smiling_face:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.