Elastic search input plugin not working

I have some data in the index of elastic search. I want to copy the data from the existing index in to the new index in elastic search using logstash. I have used elastic search input plugin and output plugin to load it to the new index.

Here's the config file i used:

input {
elasticsearch {
hosts => ["localhost:9200"]
index => "new_apps"
document_type =>"sample_apps"
query => '{ "query": {"match_all": {}} }'
size => 2000
scroll => "5m"
docinfo => true
}
}
output {
stdout {}
elasticsearch {
index => "newindex"
hosts => ["localhost:9200"]
}
}

But it's not getting data from the elastic search index. I don't know where i am doing wrong. How to fix it....... Is the configuration file is correct????

Error i got in terminal:

Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.7.0.jar) to field java.io.FileDescriptor.fd
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2019-06-07 10:43:31.437 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2019-06-07 10:43:31.458 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.8.0"}
[ERROR] 2019-06-07 10:43:35.665 [Converge PipelineAction::Create] elasticsearch - Unknown setting 'document_type' for elasticsearch
[ERROR] 2019-06-07 10:43:35.685 [Converge PipelineAction::Create] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Something is wrong with your configuration.", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/config/mixin.rb:86:in config_init'", "/usr/share/logstash/logstash-core/lib/logstash/inputs/base.rb:60:ininitialize'", "org/logstash/plugins/PluginFactoryExt.java:255:in plugin'", "org/logstash/plugins/PluginFactoryExt.java:184:inplugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:71:in plugin'", "(eval):8:ininitialize'", "org/jruby/RubyKernel.java:1061:in eval'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:ininitialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:90:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:43:inblock in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:96:in block in exclusive'", "org/jruby/ext/thread/Mutex.java:165:insynchronize'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:96:in exclusive'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:39:inexecute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:334:in `block in converge_state'"]}
[INFO ] 2019-06-07 10:43:36.323 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2019-06-07 10:43:40.905 [LogStash::Runner] runner - Logstash shut down.

Got the answer.....

Want to remove
document_type =>"sample_apps"
in input plugin. It worked great....

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.