I recently read DavidPilato's blog post : http://david.pilato.fr/blog/2015/05/20/reindex-elasticsearch-with-logstash/
Its a cool thing you can do, and for the most part it works just like David mentioned except for some problems :
-
It does not seem to work with creating the index based on a template. (I would like to copy the content from instance1_index1 and write it to instance2_index2(which is based on a template))
-
If I provide a type, it issues a warning saying the index_type declaration is deprecated, and then proceeds to spew out errors :
failed action with response of 400, dropping action: ["index", {:_id=>""cf4695d5-100c-4c8a-89f5-6c91e9ba01c8"", :_index=>"blip-2015.09.15", :_type=>"logs", :_routing=>nil}, #<LogStash::Event:0x1c82492f @metadata={"_index"=>"blip-2015.09.15", "_type"=>"twitter", "_id"=>""cf4695d5-100c-4c8a-89f5-6c91e9ba01c8"", "retry_count"=> .....
here is my config ;
input {
elasticsearch {
hosts => ["porto.int.thomsonreuters.com"]
port => "7200"
index => "blip-2015.09.15"
scroll => "5m"
docinfo => true
query => '{"query": {"filtered": {"filter": {"exists": {"field": "raw.place.bounding_box.coordinates"}}}}}'
}
}
filter {
mutate {
remove_field => ["atimestamp","@version"]
update => {"raw.place.bounding_box.type" => "multipoint" }
}
}
output {
elasticsearch {
host => "localhost"
port => 9200
protocol => "http"
index => "%{[@metadata][_index]}"
#index_type => "twitter"
document_id => "%{[@metadata][_id]}"
}
stdout {
codec => "dots"
}
}
Any ideas on if this can be made to work with index-templates ?
Thanks
Ramdev