Logstash exception for copying docs from one index to another index

Elastic Stack Logstash
1

Hi Team,

I tried to copy docs from one index to another index by using logstash.

Elasticsearch is 1.7.2 and I am using logstash 1.5.0

Below is the logstash.conf file I created.

input {

We read from the "old" index

elasticsearch {
hosts => [ "10.7.148.21" ]
port => "9200"
index => "es_item"
user => "esadmin"
password => "password"
size => 500
scroll => "5m"
docinfo => true
}
}

output {

We write to the "new" index

elasticsearch {
host => "10.7.148.21"
port => "9200"
protocol => "http"
user => "esadmin"
password => "password"
index => "es_item1"
index_type => "item"
}

We print dots to see it in action

stdout {
codec => "dots"
}
}

When I executed the below command
bin/logstash -f logstash.conf

I got the following response:-

You are using a deprecated config setting "index_type" set in elasticsearch. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"index_type", :plugin=>, :level=>:warn}

Logstash startup completed
.....................................................
....................................................
Exception in thread "Ruby-0-Thread-7: /opt/esadmin/elasticsearch/bin/logstash-1.5.0/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:92" java.lang.UnsupportedOperationException
at java.lang.Thread.stop(Thread.java:869)
at org.jruby.RubyThread.exceptionRaised(RubyThread.java:1221)
at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:112)
at java.lang.Thread.run(Thread.java:745)
Exception in thread "
at java.lang.Thread.stop(Thread.java:869)
at org.jruby.RubyThread.exceptionRaised(RubyThread.java:1221)
at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:112)
at java.lang.Thread.run(Thread.java:745)
Logstash shutdown completed

Note:- I see the docs count increase in the target index but due to the exception total docs are not copied to target index

Please guide me to fix this issue

Thanks,
Ganeshbabu R

2

Hi All,

I upgraded to logstash 1.5.5 and worked well as expected.

Regards,
Ganeshbabu R

3

Hi all,

I upgraded to ES version from 1.7.2 to 1.7.3 & installed logstash 1.5.5 and trying to copy docs from one index to another index.

Below is the sample** logstash.conf** file

input {
elasticsearch {
hosts => ["10.7.148.21:9200"]
user => "esadmin"
password => "password"
index => "item_demo"
size => 500
scroll => "5m"
docinfo=> true
}
}

filter {

}

output {
elasticsearch {
host => "10.7.148.21:9200"
protocol => "http"
user => "esadmin"
password => "password"
index => "item_test"
}
stdout {
codec => "dots"
}
}
When I executed the command bin/logstash -f logstash.conf

I got the following error response:-

failed action with response of 404, dropping action: ["index", {:_id=>nil, :_index=>"item_demo_test", :_type=>"logs", :_routing=>nil}, #<LogStash::Event:0x4eb68 @metadata_accessors=#<LogStash::Util::Accessors:0x16e82798 @store={"_index"=>"item_demo", "_type"=>"item", "_id"=>"9151301", "retry_count"=>0}, @lut={}>, @cancelled=false, @data={"ITEM_ID"=>9151301, "ITEM_CODE"=>"9151301", "ITEM_DSCR"=>"KITC HEN & TABLEWARE", "ITEM_SPECIFICITY_REF_ID"=>186, "ITEM_TYPE"=>"SGI", "IS_SHARED_IND"=>"Y", "HAS_IMAGE_IND"=>"Y", "HAS_HIST_IND"=>"Y", "MOD_CHR_VAL_ID"=>12574160, "MOD_DSCR"=>"KITCHEN & TABLEWARE", "SG_CHR_VAL_ID"=>5439993, "PG_CHR_VAL_ID"=>17428187, "ITEM_EU_NAN_CODE "=>"8378994558", "ITEM_EU_NAN_KEY"=>"4422653", "ITEM_CA_CNAN"=>nil, "ITEM_US_PRDC_ID"=>nil, "RELATIONSHIP"=>[{"REL_TYP_REF_ID"=>-999, "REL_TYPE"=>"NO RELATIONSHIP", "REL_CTGRY"=>"NIL"}], "ITEM_MISUSED_GTIN_FLG"=>"N", "CRT_DTTM"=>"2004-09-25 22:00:00", "UPD_DTTM"=>"201 4-06-01 04:01:30", "DIST"=>[{"RGN_ID"=>4, "RGN_NM"=>"DE", "DSTN_STRT_DT"=>"2007-08-22 20:24:28", "DSTN_END_DT"=>"9999-12-31 00:00:00", "ITEM_GLBL_CODE_ST_REF_ID"=>720, "ITEM_LCL_CODE_ST_REF_ID"=>720, "IS_FFU_IND"=>"Y", "FIRST_FFU_DT"=>"2012-07-06 16:17:18", "HAS_LPV_I ND"=>"N", "FIRST_DATA_DT"=>nil, "LAST_DATA_DT"=>nil, "HAS_HIST_IND"=>"Y", "LAST_CHG_DT"=>"2004-09-26 10:02:29", "IS_PREMVMNT_IND"=>"N" , "PRE_MOVEMENT_DT"=>nil}

Logstash 1.5.5 is worked well for ES version 1.7.2.

Please guide me to resolve this issue it would be very helpful.

Thanks,
Ganeshbabu R