# How to send data to Elastic Cloud with On-Premise Logstash **URL:** \ **Category:** Logstash\ **Created:** [April 17, 2020, 12:06am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423 "2020-04-17T00:06:38Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![111317](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111317/32/66249_2.png) [@111317](https://discuss.elastic.co/u/111317)\ **Post date:** [April 17, 2020, 12:06am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/1 "2020-04-17T00:06:38Z")
I am using Elastic Cloud. I want to move data from Linux on VMware to Elastic Cloud. What settings do you need? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 17, 2020, 12:10am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/2 "2020-04-17T00:10:04Z")
What sort of data? ---
**Author:** ![111317](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111317/32/66249_2.png) [@111317](https://discuss.elastic.co/u/111317)\ **Post date:** [April 17, 2020, 12:11am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/3 "2020-04-17T00:11:15Z")
Custom log in json format. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 17, 2020, 12:17am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/4 "2020-04-17T00:17:45Z")
If it's in a file then use the file input and the Elasticsearch output. Otherwise you can also try filebeat. ---
**Author:** ![111317](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111317/32/66249_2.png) [@111317](https://discuss.elastic.co/u/111317)\ **Post date:** [April 17, 2020, 12:27am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/5 "2020-04-17T00:27:52Z")
I tried it on premises and parsed it by creating a json on one line and successfully moved it as desired via the curl command, but Elastic Cloud doesn't know how to write the host part when typing curl. So I tried in Logstash but I wrote username and password and logstash.yml in Cloud-id and it didn't work. ---
**Author:** ![111317](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111317/32/66249_2.png) [@111317](https://discuss.elastic.co/u/111317)\ **Post date:** [April 17, 2020, 12:44am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/6 "2020-04-17T00:44:48Z")
The configuration settings are: ```auto input { file { path => "/path/path/*.json" start_position => "beginning" sincedb_path => "/dev/null" } } filter { json { source => "message" } } output { elasticsearch{ index =>"indexname" hosts =>["host.ap.-southeast-1.aws.found.io:9243/"] user => "username" password => "password" } } ``` I also entered cloud.id and cloud.auth in logstash.yml. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 17, 2020, 12:46am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/7 "2020-04-17T00:46:15Z")
If it didn't work, providing the logs of Logstash would help. ---
**Author:** ![111317](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111317/32/66249_2.png) [@111317](https://discuss.elastic.co/u/111317)\ **Post date:** [April 17, 2020, 12:54am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/8 "2020-04-17T00:54:25Z")
Can I upload it as an image? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 17, 2020, 12:57am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/9 "2020-04-17T00:57:28Z")
Please don't, it's very hard to read that. ---
**Author:** ![111317](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111317/32/66249_2.png) [@111317](https://discuss.elastic.co/u/111317)\ **Post date:** [April 17, 2020, 1:16am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/10 "2020-04-17T01:16:35Z")
It takes some time, please wait ---
**Author:** ![111317](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111317/32/66249_2.png) [@111317](https://discuss.elastic.co/u/111317)\ **Post date:** [April 17, 2020, 2:03am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/11 "2020-04-17T02:03:03Z")
``````auto Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option UseConcMarkSweepGC; s upport was removed in 14.0 Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option CMSInitiatingOccupanc yFraction; support was removed in 14.0 Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option UseCMSInitiatingOccup ancyOnly; support was removed in 14.0 WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (fil e:/root/logstash-7.6.2/logstash-core/lib/jars/jruby-complete-9.2.9.0.jar) to met hod sun.nio.ch.NativeThread.signal(long) WARNING: Please consider reporting this to the maintainers of com.headius.backpo rt9.modules.Modules WARNING: Use --illegal-access=warn to enable warnings of further illegal reflect ive access operations WARNING: All illegal access operations will be denied in a future release Sending Logstash logs to /root/logstash-7.6.2/logs which is now configured via log4j2.properties [2020-04-17T07:11:12,944][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified [2020-04-17T07:11:13,087][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.6.2"} [2020-04-17T07:11:15,342][INFO][org.reflections.Reflections] Reflections took 45 ms to scan 1 urls, producing 20 keys and 40 values [2020-04-17T07:11:15,800][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (ArgumentError) URI is not valid - host is not specified", :backtrace=>["org.logstash.config.ir.CompiledPipeline.(CompiledPipeline.java:103)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:60)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1156)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "root.logstash_minus_7_dot_6_dot_2.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/root/logstash-7.6.2/logstash-core/lib/logstash/java_pipeline.rb:27)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:84)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.java:915)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "root.logstash_minus_7_dot_6_dot_2.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/root/logstash-7.6.2/logstash-core/lib/logstash/pipeline_action/create.rb:36)", "root.logstash_minus_7_dot_6_dot_2.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$ __VARARGS__ (/root/logstash-7.6.2/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:84)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "root.logstash_minus_7_dot_6_dot_2.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/root/logstash-7.6.2/logstash-core/lib/logstash/agent.rb:326)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:71)", "org.jruby.runtime.Block.call(Block.java:125)", "org.jruby.RubyProc.call(RubyProc.java:274)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.base/java.lang.Thread.run(Thread.java:832)"]} warning: thread "Converge PipelineAction::Create
" terminated with exception (report_on_exception is true): LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create
` create at org/logstash/execution/ConvergeResultExt.java:109 add at org/logstash/execution/ConvergeResultExt.java:37 converge_state at /root/logstash-7.6.2/logstash-core/lib/logstash/agent.rb:339 [2020-04-17T07:11:15,811][ERROR][logstash.agent] An exception happened when converging configuration {:exception=>LogStash::Error, :message=>"Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create
`", :backtrace=>["org/logstash/execution/ConvergeResultExt.java:109:in `create'", "org/logstash/execution/ConvergeResultExt.java:37:in `add'", "/root/logstash-7.6.2/logstash-core/lib/logstash/agent.rb:339:in `block in converge_state'"]} [2020-04-17T07:11:15,893][FATAL][logstash.runner] An unexpected error occurred! {:error=>#`>, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:109:in `create'", "org/logstash/execution/ConvergeResultExt.java:37:in `add'", "/root/logstash-7.6.2/logstash-core/lib/logstash/agent.rb:339:in `block in converge_state'"]} [2020-04-17T07:11:15,918][ERROR][org.logstash.Logstash] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit ````` `````` ---
**Author:** ![111317](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111317/32/66249_2.png) [@111317](https://discuss.elastic.co/u/111317)\ **Post date:** [April 17, 2020, 2:03am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/12 "2020-04-17T02:03:45Z")
Sorry I'm late. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 17, 2020, 2:28am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/13 "2020-04-17T02:28:47Z")
> [@111317](#): > > Unable to configure plugins: (ArgumentError) URI is not valid - host is not specified" Try adding `https://` to the front of your `hosts` value and see if that helps. ---
**Author:** ![111317](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111317/32/66249_2.png) [@111317](https://discuss.elastic.co/u/111317)\ **Post date:** [April 17, 2020, 3:29am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/14 "2020-04-17T03:29:55Z")
Thank you, but it worked, but it never arrived in the cloud. ---
**Author:** ![111317](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111317/32/66249_2.png) [@111317](https://discuss.elastic.co/u/111317)\ **Post date:** [April 17, 2020, 3:49am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/15 "2020-04-17T03:49:45Z")
```auto Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option UseConcMarkSweepGC; support was removed in 14.0 Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option CMSInitiatingOccupancyFraction; support was removed in 14.0 Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option UseCMSInitiatingOccupancyOnly; support was removed in 14.0 WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/root/logstash-7.6.2/logstash-core/lib/jars/jruby-complete-9.2.9.0.jar) to method sun.nio.ch.NativeThread.signal(long) WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Sending Logstash logs to /root/logstash-7.6.2/logs which is now configured via log4j2.properties [2020-04-17T08:58:32,336][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified [2020-04-17T08:58:32,470][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.6.2"} [2020-04-17T08:58:34,506][INFO][org.reflections.Reflections] Reflections took 49 ms to scan 1 urls, producing 20 keys and 40 values [2020-04-17T08:58:35,610][INFO][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elastic:xxxxxx@a77aaf4882664376b700d79c0fa670c6.ap-southeast-1.aws.found.io:9243/]}} [2020-04-17T08:58:36,529][WARN][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"https://elastic:xxxxxx@a77aaf4882664376b700d79c0fa670c6.ap-southeast-1.aws.found.io:9243/"} [2020-04-17T08:58:37,026][INFO][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7} [2020-04-17T08:58:37,032][WARN][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7} [2020-04-17T08:58:37,259][INFO][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://a77aaf4882664376b700d79c0fa670c6.ap-southeast-1.aws.found.io:9243/"]} [2020-04-17T08:58:37,350][WARN][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge][main] A gauge metric of an unknown type (org.jruby.RubyArray) has been created for key: cluster_uuids. This may result in invalid serialization. It is recommended to log an issue to the responsible developer/development team. [2020-04-17T08:58:37,360][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, "pipeline.sources"=>["/root/logstash-7.6.2/config/test.conf"], :thread=>"#"} [2020-04-17T08:58:37,471][INFO][logstash.outputs.elasticsearch][main] Using default mapping template [2020-04-17T08:58:37,756][INFO][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}} [2020-04-17T08:58:38,763][INFO][logstash.javapipeline][main] Pipeline started {"pipeline.id"=>"main"} [2020-04-17T08:58:38,839][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]} [2020-04-17T08:58:38,867][INFO][filewatch.observingtail][main] START, creating Discoverer, Watch with file and sincedb collections [2020-04-17T08:58:39,163][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600} ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 15, 2020, 3:49am UTC](https://discuss.elastic.co/t/how-to-send-data-to-elastic-cloud-with-on-premise-logstash/228423/16 "2020-05-15T03:49:48Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.