Logstash를 통해 Elastic Cloud로 데이터를 전송하는 방법

input {
	file {
	  path => "/path/path/*.json"
	  start_position => "beginning"
	  sincedb_path => "/dev/null"
}
}
filter {
	json {
	     source => "message"
}
}
output {
	elasticsearch{
		index =>"indexname"
		hosts =>["host.ap.-southeast-1.aws.found.io:9243/"]
		user => "username"
		password => "password"
}
}

이런식으로 작성했는데 안됩니다. logstash.yml에도 cloud.id와 cloud.auth 입력했는데도요 Java.Lang오류 뜨면서 안되네요

Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option UseConcMarkSweepGC; s                                                                               upport was removed in 14.0
Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option CMSInitiatingOccupanc                                                                               yFraction; support was removed in 14.0
Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option UseCMSInitiatingOccup                                                                               ancyOnly; support was removed in 14.0
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (fil                                                                               e:/root/logstash-7.6.2/logstash-core/lib/jars/jruby-complete-9.2.9.0.jar) to met                                                                               hod sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backpo                                                                               rt9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflect                                                                               ive access operations
WARNING: All illegal access operations will be denied in a future release
Sending Logstash logs to /root/logstash-7.6.2/logs which is now configured via log4j2.properties
[2020-04-17T07:11:12,944][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-04-17T07:11:13,087][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.6.2"}
[2020-04-17T07:11:15,342][INFO ][org.reflections.Reflections] Reflections took 45 ms to scan 1 urls, producing 20 keys and 40 values
[2020-04-17T07:11:15,800][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (ArgumentError) URI is not valid - host is not specified", :backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:103)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:60)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1156)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "root.logstash_minus_7_dot_6_dot_2.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/root/logstash-7.6.2/logstash-core/lib/logstash/java_pipeline.rb:27)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:84)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.java:915)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "root.logstash_minus_7_dot_6_dot_2.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/root/logstash-7.6.2/logstash-core/lib/logstash/pipeline_action/create.rb:36)", "root.logstash_minus_7_dot_6_dot_2.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$__VARARGS__(/root/logstash-7.6.2/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:84)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "root.logstash_minus_7_dot_6_dot_2.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/root/logstash-7.6.2/logstash-core/lib/logstash/agent.rb:326)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:71)", "org.jruby.runtime.Block.call(Block.java:125)", "org.jruby.RubyProc.call(RubyProc.java:274)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.base/java.lang.Thread.run(Thread.java:832)"]}
warning: thread "Converge PipelineAction::Create<main>" terminated with exception (report_on_exception is true):
LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`
          create at org/logstash/execution/ConvergeResultExt.java:109
             add at org/logstash/execution/ConvergeResultExt.java:37
  converge_state at /root/logstash-7.6.2/logstash-core/lib/logstash/agent.rb:339
[2020-04-17T07:11:15,811][ERROR][logstash.agent           ] An exception happened when converging configuration {:exception=>LogStash::Error, :message=>"Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`", :backtrace=>["org/logstash/execution/ConvergeResultExt.java:109:in `create'", "org/logstash/execution/ConvergeResultExt.java:37:in `add'", "/root/logstash-7.6.2/logstash-core/lib/logstash/agent.rb:339:in `block in converge_state'"]}
[2020-04-17T07:11:15,893][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`>, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:109:in `create'", "org/logstash/execution/ConvergeResultExt.java:37:in `add'", "/root/logstash-7.6.2/logstash-core/lib/logstash/agent.rb:339:in `block in converge_state'"]}
[2020-04-17T07:11:15,918][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

안녕하세요
에러 메시지가 나와있는데 살펴보셨나요?

여기 참고해 보시구요
https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-hosts

지금
host.ap.-southeast-1.aws.found.io:9243/
이게 유효한 URI인지 확인해 보시면 좋을 듯 하네요

저 호스트는 유효합니다 curl을 통해서 보내봤습니다. host는 공개해도 되는 내용인지 몰라서 host라고 써놓은 겁니다.

input { file } 에서

sincedb_path => "/dev/null"

이 설정을 적용하려면 Logstash 가 sudo 로 실행되어야 합니다. 권한 한번 확인 해 보세요.
추가적으로 Java 도 지원하는 버전이 설치되어 있는지 확인 해 보시기 바랍니다.

https://www.elastic.co/support/matrix#matrix_jvm

말씀하신대로 sudo로 사용해보았습니다. 역시나 안됩니다. java 도 openjdk 1.8.0버전을 사용하고 있습니다.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.